Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Assessment in a tribal college context : a case study of Northwest Indian College Karlberg, Anne Marie 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_2007-317969.pdf [ 19.04MB ]
JSON: 831-1.0055897.json
JSON-LD: 831-1.0055897-ld.json
RDF/XML (Pretty): 831-1.0055897-rdf.xml
RDF/JSON: 831-1.0055897-rdf.json
Turtle: 831-1.0055897-turtle.txt
N-Triples: 831-1.0055897-rdf-ntriples.txt
Original Record: 831-1.0055897-source.json
Full Text

Full Text

ASSESSMENT IN A TRIBAL C O L L E G E CONTEXT: A C A S E STUDY OF NORTHWEST INDIAN C O L L E G E  by ANNE MARIE KARLBERG  B.Sc. (Honours), The University o f Toronto, 1986 M . P . H . , Tulane University, 1989 B . E d . , The University of Toronto, 1992  A THESIS S U B M I T T E D I N P A R T I A L F U L F I L M E N T OF THE REQUIREMENTS F O R T H E D E G R E E OF D O C T O R OF PHILOSOPHY in  THE F A C U L T Y OF G R A D U A T E STUDIES  (Educational Studies)  THE U N I V E R S I T Y OF BRITISH C O L U M B I A July 2007 © Anne Marie Karlberg, 2007  ABSTRACT  Approximately 32 tribal colleges are located on reservations i n the United States. Their aim is to provide Native American students with a culturally relevant and meaningful postsecondary education. Assessment uses methods o f applied research to improve student learning. The aim o f this study is to advance theoretical and applied knowledge in the field o f assessment within tribal colleges. This undertaking is noteworthy given that tribal colleges are vital to the development and future o f Native American communities. I use a case study methodology to examine a specific assessment program that is being developed at Northwest Indian College ( N W I C ) , a tribal college i n Washington state. In this study, I provide responses to three research questions: (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) W h i c h elements of the N W I C assessment program are most and least successful according to the evaluative criteria established in Research Question 1? and (3) What preconditions and other contextual factors contribute to the relative success or failure o f different elements o f the N W I C assessment program? I review the history o f Native Americans in higher education, provide an overview and critique o f the emerging assessment movement, and discuss the state o f assessment within the tribal college system. This study demonstrates that assessment can be done i n a tribal college context in a culturally respectful and meaningful way and provides insights into how this can be approached. M y findings suggest that effective tribal college assessment programs use three types o f information to assess student learning — direct indicators, indirect indicators, and institutional and community data — and assess each o f these at the tribal community, college, program, and course levels. It is equally important to pay attention to the inputs into the assessment program  (i.e., plans and resources) and how the assessment program is carried out. Furthermore, the intention o f the assessment program is to impact the college's context. Articulating and revisiting the tribal college's mission — with its focus on the self-determination o f Native peoples — is a critical initial step in the development o f assessment programs that should be emphasized i n this framework.  T A B L E  O F  CONTENTS  A B S T R A C T T A B L E  ii  O F CONTENTS  iv  LIST O F T A B L E S  viii  LIST O F FIGURES  ix  A C K N O W L E D G E M E N T S  x  C H A P T E R  1  1 I N T R O D U C T I O N  Research Questions N W I C Assessment Program Case Study Positioning M y s e l f Significance o f Organizational Insider Research Overview o f the Doctoral Thesis C H A P T E R 2 N A T I V E A M E R I C A N S O V E R V I E W  IN H I G H E R EDUCATION  3 3 6 9 11 — HISTORY  O F T R I B A L C O L L E G E S  13  The History o f Native Americans i n Higher Education in the United States The Founding o f Tribal Colleges i n the United States (1960s) Tribal Colleges Today Conclusion C H A P T E R 3 ASSESSMENT IN T H EUNITED STATES — O V E R V I E W CRITIQUE  What is Assessment? Why do Assessment? Criticisms and Concerns about Assessment Benefits o f Effective Assessment Processes Mainstream Approaches to Assessment Types o f Information U s e d for Assessment Purposes Assessment Issues Specific to Two-Year Colleges and Tribal Colleges Emerging Innovative Approaches to Assessment Planning and Reporting Conclusion C H A P T E R 4 ASSESSMENT IN T R I B A L C O L L E G E S  Guiding Principles State o f Assessment in Tribal Colleges Conclusion  A N D  14 31 38 58 A N D 60  61 63 66 81 82 84 97 100 105 109 110  110 116 137  iv  C H A P T E R 5 M E T H O D O L O G Y  139  Case Studies The C I P P M o d e l Research Design Respondents Data Analysis C H A P T E R 6 E V A L U A T I O N  139 147 150 151 152  ;  F R A M E W O R K A N D R E S U L T S  153  Context (Research Question 3) Inputs (Research Questions 1 and 2) Process (Research Questions 1 and 2) Products (Research Questions 1 and 2) Conclusion  153 156 171 180 210  C H A P T E R 7 CONCLUSIONS  211  Response to Research Question 1 (Criteria) Response to Research Question 2 (Most and Least Successful Elements) Response to Research Question 3 (Preconditions and Contextual Factors) Implications o f Research Limitations o f the Study Suggestions for Future Research and W o r k Concluding Remarks  212 219 224 228 240 242 243  R E F E R E N C E S APPENDIX A  245 S E C T I O N O F T H E N W I C A S S E S S M E N T P L A N ( D R A F T ) (2007 - 2012) 263  Introduction Inputs Processes Products  263 264 267 271  APPENDIX B  S U R V E Y O F T E A C H I N G  A N D ASSESSMENT M E T H O D S  APPENDIX C  N W I C ASSESSMENT S U R V E Y  APPENDIX D  N W I C INSTITUTIONAL R E V I E W  (2003)  288 289  B O A R D L E T T E R O F A P P R O V A L 304  APPENDIX E  U B CB E H A V I O U R A L  R E S E A R C H ETHICS B O A R D ' S  CERTIFICATE  OF A P P R O V A L APPENDIX F  S U M M A R Y O F C L O S E D - E N D E D ASSESSMENT  RESPONSES APPENDIX G  305 S U R V E Y :.  MISSION S T A T E M E N T  Results, Challenges, and Strengths Suggestions and Actions  306 312  312 313  v  APPENDIX H  ADMINISTRATORS' R O L E S  Results, Challenges, and Strengths Suggestions and Actions APPENDIX I  F A C U L T Y M E M B E R S ' R O L E S  Results, Challenges, and Strengths Suggestions and Actions APPENDIX J ASSESSMENT P R O C E S S  Results, Challenges, and Strengths Suggestions and Actions APPENDIX K  E M B E D D I N G ASSESSMENT IN C O L L E G E P R O C E S S E S  Results, Challenges, and Strengths Suggestions and Actions APPENDIX L  L E A R N I N G , T E A C H I N G , A N DA S S E S S M E N T A P P R O A C H E S  Results, Challenges, and Strengths Suggestions and Actions APPENDIX M  C O L L E G E O U T C O M E S INITIATIVE  Results, Challenges, and Strengths Suggestions and Actions APPENDIX N  C U L T U R A L O U T C O M E S INITIATIVE  Results, Challenges, and Strengths Suggestions and Actions APPENDIX O  C O U R S E O U T C O M E S INITIATIVE  Results, Challenges, and Strengths Suggestions and Actions APPENDIX P INDIRECT INDICATORS  Results, Challenges, and Strengths Suggestions and Actions APPENDIX Q  INSTITUTIONAL D A T A  Results, Challenges, and Strengths Suggestions and Actions APPENDIX R N W I C RUBRICS  Written Communication Rubric (Draft) Oral Communication Rubric (Draft) Computer Skills Rubric (Draft) Quantitative Skills Rubric (Draft) Reading Skills Rubric (Draft)  316  316 317 318  318 318 319  319 320 321  321 322 324  324 325 328  328 329 330  330 331 332  332 333 334  334 335 336  336 337 339  339 341 343 344 345  vi  APPENDIX S E X A M P L E O F A C O M P L E T E D N W I C C O U R S E O U T C O M E S  F O R M 346  APPENDIX T  N E E D S A S S E S S M E N T (2003)  348  APPENDIX U  S T U D E N T O P I N I O N S U R V E Y — E X T R A Q U E S T I O N S (2004)  351  APPENDIX V  A L U M N I S U R V E Y — E X T R A Q U E S T I O N S (2003)  354  APPENDFX W APPENDIX X  A N E X A M P L E O F A N N W I C C O U R S E E V A L U A T I O N F O R M N W I C R E T E N T I O N STRATEGIES  358 361  A P P E N D I X Y C H E C K L I S T F O R T R I B A L C O L L E G E A S S E S S M E N T P R O G R A M S . 363  Inputs Processes Products A P P E N D I X Z C O V E R L E T T E R F O RT H EN W I C A S S E S S M E N T S U R V E Y  363 365 367 370  vii  LIST OF TABLES  Table 3.1 Examples o f Direct Methods for Assessing Student Learning  86  Table 3.2 A Section o f the N W I C Rubric for Evaluating Writing Skills (Draft)  89  Table 3.3 M e a n Aggregate Scores from a Native American History Class Assignment Assessing the N W I C Outcome "Writing Standard English" 90 Table 3.4 Relationship between Individual Student Grading and Assessment  91  Table 3.5 Example o f a Section of a Curriculum M a p  93  Table 3.6 Examples o f Indirect Methods for Assessing Student Learning  95  Table 3.7 Examples o f Institutional Data  96  Table 3.8 Example o f a Section o f an Environmental Studies Program Assessment Plan  108  Table 5.1 List o f Research Questions, Data Collection Methods, and CIPP Areas  149  Table 5.2 Demographic Breakdown o f Staff who Responded to the Survey  151  Table 6.1 Draft List o f Faculty Cultural Outcomes  189  Table 7.1 Examples o f Activities at Each Level o f a Tribal College Assessment Program  239  viii  LIST O F FIGURES  Figure 2.1 M a p o f Tribal Colleges and Universities i n the United States  41  Figure 6.1 Time-line o f Assessment Activities at N W I C (Products of the Assessment Program) 182 Figure 7.1 Conceptual M o d e l of the Components o f a Tribal College Assessment Program... 218 Figure 7.2 Simplified Conceptual M o d e l o f the Components o f a Tribal College Assessment Program 231 Figure 7.3 Examples of Products of a Tribal College Assessment Program  236  ix  ACKNOWLEDGEMENTS  This thesis is the culmination o f years o f work, learning, and growth, and it would never have been possible without the support o f many people. I wish to acknowledge with sincere thanks my committee members, Lesley Andres, Jo-ann Archibald, and T o m Sork, who were nothing but encouraging from the start. The dedicated and outstanding staff at Northwest Indian College are a daily inspiration to me. I am especially appreciative for the ongoing interest and support o f President Cheryl Crazy B u l l , Rissa Wabaunsee, Ted Williams, Barbara Roberts, and Sharon Kinley, all o f whom found time in their busy schedules to review the thesis and provide valuable insights. A l s o , thank you to my colleagues at Northwest Indian College - Brian Compton, B i l l Freeman, and E l v a Eisel - and to Greg Gagnon from the University o f North Dakota for reviewing the survey and thesis and providing helpful feedback. Thank you to all the faculty, administrators, and staff for their ongoing support of, and perseverance with, the assessment, teaching, and learning work. I appreciate the friendships from my Educational Studies cohort and especially the constant moral support from Michelle Pidgeon. For the editing o f this thesis, I am most grateful to Kathy Roesch, whose input has enriched significantly its quality. To m y mother, Marcia Kuhlberg, the most honourable person I know, I am proud to be your daughter. Michael, thank you for believing in this project. Y o u r insight and capacity continually amaze me. To our beautiful and talented daughters, Jessa and Leah, thank you for your patience. I love you more than you can imagine. A n d finally, to all tribal college students: This work is ultimately for you.  CHAPTER 1 INTRODUCTION  This study examines assessment in a Native American tribal college context. Tribal colleges are located on reservations in the United States. Their aim is to provide primarily Native American students with a culturally relevant and meaningful post-secondary education. Assessment uses methods o f applied research to improve a college's mission-related performance — especially as it relates to student learning; however, Native American theory and research protocols suggest that, to date, the type o f data and methods o f data collection that have characterized most mainstream assessment are not well-suited to a Native American context (Battiste & Henderson, 2000; Boyer, 2003; Carnegie Foundation for the Advancement o f Teaching, 1997; Haig-Brown & Archibald, 1996; Hampton, 1995b; Kirkness & Barnhardt, 1991; Menzies, 2001; National Sciences and Engineering Research Council o f Canada, 1999; R o y a l Commission, 1996; L . Smith, 1999). Fortunately, approaches to assessment are changing and more progressive assessment programs are now becoming valuable tools in higher education throughout the United States — even though they are not yet well developed in Native American contexts. In fact, a study o f American tribal colleges by the Carnegie Foundation for the Advancement o f Teaching (Carnegie, 1997) determined that, due to the lack of. financial resources, expertise, and technology, even the most basic assessment data are generally not available in tribal colleges. Nonetheless, Indigenous educators are beginning to recognize assessment as an emerging priority — for internal improvement as well as for accreditation purposes (Blanchard et al., 2000; Ortiz & HeavyRunner, 2003; L . Smith, 1999; Swisher & Tippeconnic, 1999). The W o r l d Indigenous Nations Higher Education Consortium (2003), which "provides an international forum and support for Indigenous Peoples to pursue common goals through higher education" (p.  1  1), has the goal o f creating "an accreditation body for Indigenous education initiatives and systems that identify common criteria, practices, and principles by which Indigenous Peoples live" (p. 1). The general concern among Native American administrators is that i f tribal colleges do not articulate for themselves a meaningful approach to assessment, then external accreditation bodies w i l l do it for them. Elaborating on this theme, Swisher (Hunkpapa Lakota), president o f Haskell Indian Nations University i n Lawrence, Kansas, and Tippeconnic (Comanche), professor o f education at The Pennsylvania State University and Director o f the American Indian Leadership Program, explain that assessment research i n tribal colleges should revolve around teaching, curricula, pedagogy, and higher education issues such as "enrollment, retention, and graduation" (1999, p. 298). They state that Indigenous institutions should develop "alternative assessment or unbiased standardized tests to assess student achievement and abilities" (p. 298) as they can play a valuable role in decision making to improve practice. Demmert (2005), an Alaska Tlingit Native and professor at Western Washington University's College o f Education, also comments on the need to focus on developing "measures for assessing e.g., cognitive development, language development, academic achievement, for identifying those who need interventions, and for the monitoring o f student progress" (p. 21). He states that "new measures that are culturally and, where applicable, linguistically appropriate (i.e., in Native languages)" and that "could be researcher-developed or could be produced by test publishers.. .are specifically needed that would address culture-based education" (p. 21). Finally, he says that educators in Indigenous educational institutions can "examine existing approaches to program evaluation and incorporate evaluation in the planning and implementation o f any new (or existing but unevaluated) programs. The fruits o f any such program evaluations should be  2  shared broadly, v i a a central clearinghouse, publications, or v i a any networks or list-serves that exist" (p. 22). In this context, the primary aim o f this study is to advance theoretical and applied knowledge in the field o f assessment, within tribal colleges. In 2002, administrators and faculty at Northwest Indian College ( N W I C ) , a tribal college on the L u m m i Reservation i n northwestern Washington state, asked me to be their assessment coordinator and to assist them i n devising and implementing an assessment program that is appropriate and meaningful to their tribal college mission. For my doctoral research, I present the context o f N W I C s assessment program and then document and critically evaluate the inputs, processes, and products o f this innovative assessment effort, so that other tribal colleges can learn from our experience.  Research Questions  Through this research, I seek to answer the following questions: (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) W h i c h elements o f the N W I C assessment program are most and least successful according to the evaluative criteria established in Research Question 1 ? (3) What preconditions and other contextual factors contribute to the relative success or failure o f different elements o f the N W I C assessment program?  N W I C Assessment P r o g r a m Case Study  I carried out this research at N W I C on the L u m m i Reservation in Washington state. N W I C is 1 o f approximately 32 tribal colleges throughout the United States that prepare Native  3  American students for positions o f leadership and self-determination within their communities, through culturally appropriate pedagogy and curricula.  The L u m m i Reservation The L u m m i Reservation occupies approximately 12,500 acres o f land on a peninsula that extends into the Pacific Ocean in the northwestern corner o f Washington state (Lummi Indian Business Council, 1993). It is located just a 2-hours' drive south o f Vancouver, British Columbia, and a 2-hours' drive north o f Seattle. There are approximately 3,800 enrolled Lummis, with about 2,100 Lummis currently living on the reservation (U.S. Census Bureau, 2000). The L u m m i are part o f the Coast Salish cultural and linguistic group, and their main sources o f employment are fishing, gathering shellfish, the gaming industry, agriculture, and forestry.  The History o f Higher Education on the L u m m i Reservation Higher education on the L u m m i Reservation is rooted i n the community's desire to provide its older youth and adults with an educational program that reflects and respects L u m m i ' s cultural heritage, knowledge, customs, and needs. A s a result, in 1973, the L u m m i Reservation took the first step i n this direction by establishing the L u m m i Indian School o f Aquaculture. This school was designed to train Lummis and other Native Americans i n the technical aspects o f operating fish and shellfish hatcheries. Trained technicians went on to work in Native American owned and operated hatcheries throughout the United States and Canada. B y the early 1980s, however, demand for these technicians declined drastically. A t about the same time, the high rate o f general unemployment and the low rate o f high school completion on the reservation created an awareness o f the need for a post-secondary tribal  4  college in northwest Washington; therefore, i n 1983, the L u m m i Indian Business Council closed the L u m m i Indian School o f Aquaculture and established a public, non-profit community college. The purpose o f the college was to provide adult basic education (i.e., basic reading, writing, math, and life skills) and post-secondary education to northwest Washington Native American communities (Hayes, 1990; N W I C , 2002). Thus the L u m m i Nation established L u m m i Community College i n 1983 as a Native American administered community college. Initially, L u m m i Community College operated under a contract with Whatcom Community College, which is an accredited college nearby. Within the scope o f this contract, courses offered at L u m m i Community College had to meet standards set by Whatcom Community College for maintaining its own accreditation ( N W I C , 1991). In 1988, the L u m m i Community College charter was expanded to include higher educational opportunities for tribes i n Washington, Oregon, and Idaho. Because members o f various Northwest tribes attended the college, a more representative name for the college was desired. In 1989, it began operating independently o f Whatcom Community College and, in order to more accurately reflect its new mission and the wider communities it now served, its ~ name was officially changed to Northwest Indian College ( N W I C , 2002).  N W I C Today N W I C is a tribally administered institution chartered by the L u m m i Indian Business Council. Although the college predominantly relies on grants for its funding, it is also federally funded through the Bureau o f Indian Affairs. In 1993, the Northwest Association o f Schools and Colleges accredited N W I C ( N W I C , 2002). N W I C is the only accredited tribal college in Washington, Oregon, and Idaho. Although the college primarily serves four northwestern states (Washington, Oregon, Idaho, and Alaska), some o f its 650 students come from all corners o f the  5  United States, as well as from Canada. N W I C has six extended campuses — on the Colville, Muckleshoot, N e z Perce (Idaho), Port Gamble S ' K l a l l a m , Swinomish, and Tulalip reservations — which offer face-to-face classes and classes transmitted on interactive television (ITV) from the main L u m m i campus. Interactive television involves students attending classes at extended campuses while having a live connection to the main campus class. In addition, a significant portion o f the college enrollment is generated through extension services, such as independent learning. N W I C is located on the old site o f the L u m m i D a y School, built i n 1910. The original L u m m i Day School building now houses the college's library. The college campus consists o f mainly portable structures. Although the current president, the vice presidents, and most o f the staff are Native American, most of the faculty members are not. Crazy B u l l (2006a), N W I C ' s president, summarizes her expectations for the college: W e are a place where the vision o f our ancestors comes together with the present — our students today — to build a future for coming generations. W e are the place where our understanding o f traditions and cultural practice are translated into contemporary education. W e are the place where Native knowledge is honored, new leadership blossoms, and where students find their voice, (p. 1) Thus, N W I C strives to provide a holistic approach to education i n a culturally relevant and supportive environment that provides students with as many opportunities for success as possible. Its mission statement emphasizes the college's hope: "Through education, Northwest Indian College promotes Indigenous self-determination and knowledge."  Positioning M y s e l f In February 2002, N W I C hired me to coordinate its assessment program. Since that time, I have been facilitating and coordinating a participatory assessment process related to teaching and learning at the college. In this regard, I am a cultural outsider doing organizational insider  6  research. A s a White woman, from a relatively privileged background, my understanding o f the world has been shaped by my own culture, education, and life experiences. A s a result, I am challenged to continually reflect on how my own worldviews impact my research and assessment work within a tribal college context (Haig-Brown, 1992). If I am going to be involved in creating knowledge that relates to Indigenous people, I must situate myself as a learner within the tradition o f Indigenous knowledge (Mclsaac, 2000). From this position, I can learn much both personally and professionally (Haig-Brown, 1992) while, at the same time, I also hope that through a participatory research process, I can assist in generating new knowledge that directly serves N W I C and from which other tribal colleges can benefit. M a n y Indigenous scholars have articulated their thoughts regarding non-Natives working cooperatively within Indigenous communities and educational institutions. Linda Tuhiwai Smith (2000), professor o f M a o r i Education and Director o f the International Research Institute for M a o r i and Indigenous Studies at the University o f Auckland, N e w Zealand, states that nonNatives need not be excluded from contributing to Native American educational research. Mihesuah (2004), editor o f the American Indian Quarterly, echoes this thought and believes that "a balanced, inclusive methodology is ideal" (p. xi). Graham Smith (1992), Visiting Distinguished Professor in the Department o f Educational Studies graduate program at the University o f British Columbia and a M a o r i education scholar, articulates situations in which it may be appropriate for non-Indigenous researchers to conduct research i n Indigenous communities. One o f these is i f the researcher empowers the community to address issues and concerns in a manner that leads to beneficial outcomes. Furthermore, Marker (2000), Director o f T s " k e l First Nations Graduate Studies in Education at the University o f British Columbia, states clearly that "the quality o f research in First Nations education is not improved simply by having  7  Aboriginal people doing the writing. It is improved by a more detailed analysis that includes the perspectives and location o f both Natives and non-Natives" (p. 31). This is the perspective that the N W I C administration adopted when they hired me as their assessment coordinator in 2002. In their 1998 accreditation review, administrators at N W I C received several recommendations regarding assessment that were essential to address before their next review. The following recommendations related to assessment: (1) The team recommends that the college move deliberately to a process o f broad-based continuous planning and evaluation (Standard I.B). (2) The team recommends that the college adopt and consistently implement an educational assessment plan, which provides for a series o f outcomes measures that are internally consistent and in accord with its mission (Standard 2; Policy 2.2). (3) The team recommends that the college adopt a consistent systematic process o f evaluation o f faculty, staff, and administration (Standards 3.A.2, 6.C.3; policy 4.1). (Northwest Commission o f Colleges and Universities, 1998)  The administrators decided the assessment work was worthwhile and i n their best interest, but that it should be done in a culturally meaningful way. A s a result, the administrators' primary interest was to find someone who could respect their cultural protocols, beliefs, and values and integrate them with conventional research procedures and best practices (Marker, 2004; L . Smith, 1999; Steinhauer, 2002). Unable to find or attract a Native person with the required skills and expertise in this regard, they accepted me as a partner i n their assessment work. Just as some non-Natives have contributed positively to Native American struggles regarding control over education (Haig-Brown, 1992), I have now become an ally linking N W I C to relevant resources and expertise in the non-Native world. Through m y work at N W I C , I also have opportunities to educate non-Native communities o f interest, such as accreditation bodies and funding agencies, and to translate Native values and interests into language that is familiar to  8  these external agencies. Nonetheless, I can conduct this research only because I have been invited into and accepted within this community and have been asked to perform this work. Moreover, the literature on research within Native American communities emphasizes the importance o f the commitment o f researchers to a participatory process, based on a respectful and reciprocal relationship between the researcher and the community (Haig-Brown & Archibald, 1996; Hampton, 1995b; Kirkness & Barnhardt, 1991; Menzies, 2001; National Sciences and Engineering Research Council o f Canada, 1999; Royal Commission on Aboriginal Peoples, 1996; L . Smith, 1999). In addition, as alluded to above, the research itself needs to be relevant and meaningful to the community and conducted i n a responsible and ethical manner (Darou, Kurtness, & H u m , 2000; Greenwood & L e v i n , 1998; Royal Commission, 1996; L . Smith, 1999; Stringer, 1999). Ideally, these principles should guide research performed in any community, but past experience within Native American communities has left them acutely sensitive to abuses in this regard. In m y role as a bridge between N W I C and the field o f assessment, it is especially important that I introduce and facilitate approaches to assessment that are consistent with these research principles. N W I C administrators have been extremely encouraging and supportive o f this doctoral research. In the following section, I briefly discuss the relevance o f insider research and clarify my own position in this regard.  Significance of Organizational Insider Research  Various approaches to organizational insider research — such as participative inquiry, cooperative inquiry, participatory action research, and action inquiry — have been developed in the past decade as this practice has been increasingly legitimized (Bensimon, Polkinghorne, Bauman, & Vallejo, 2004). A l l o f these approaches involve the community identifying a problem or concern and the researcher collaborating with the community in order to address the  9  problem or concern through a participatory research process. A s a form o f action research, assessment connects insider research to pedagogical practice. B y deriving assessment research from community-identified needs, by engaging the community as participants in the research process, and by reporting the results back to the community in meaningful ways, Bensimon et al. (2004) contend practical knowledge is produced. In this regard, they refer to a "practitioner-asresearcher" (p. 109) model, a method o f research for bringing about college-wide change, i n which "stakeholders produce the knowledge" (p. 109) (instead o f external researchers) to provide information that illuminates local problems i n a specific context in order to work toward a solution; thus the insiders conduct the research and produce the information. The benefits o f organizational insider research are now well documented (L. Smith, 1999). For instance, by interviewing key people and being a participant-observer, insiders can effectively draw on their experiences and observations within the community as well as use "reflective thinking" to enhance their research (Ffaig-Brown, 1992; Hampton, 1995b; Pelto & Pelto, 1978). Within a college context, this information can be used to improve institutional performance and generate practical information that is effective for initiating change (Bensimon et al., 2004); however, as L i n d a Smith (1999) emphasizes, "the critical issue with insider research is the constant need for reflexivity" (p. 137). Reflexivity refers to the ability to reflect fairly upon one's actions and responsibilities and to respond continually to changing circumstances and needs. In a tribal college context, involving faculty, staff, students, and community members i n this reflective process can improve learning and teaching and ensure that assessment processes are serving the college effectively (Stufflebeam, 2003a). To do this, however, inside researchers need, among other things, to train participants in assessment techniques, help them acquire or develop assessment resources, seek feedback on draft  10  assessment reports, facilitate the use o f relevant information in databases, and assist in creating realistic assessment plans (Stufflebeam, 2003a).  O v e r v i e w of the Doctoral Thesis  The doctoral thesis is divided into seven chapters. In chapter 1,1 have established the research questions and positioned myself as a non-Native person working at a tribal college and participating in organizational insider research that affects this community. In addition, this chapter introduced N W I C as a case study. It provided an overview o f the L u m m i Reservation; reviewed the history o f higher education on the L u m m i Reservation; and examined where N W I C is today. Chapter 2 provides a summary o f Native Americans in higher education, including a history and overview o f tribal colleges i n the United States. Chapter 3 provides an overview and critique o f contemporary approaches to assessment. Chapter 4 is divided into two sections. The first section summarizes guiding principles o f culturally appropriate research in Native American communities that are most relevant to the future o f assessment at tribal colleges. The second section reviews emerging assessment practices and discussions i n tribal colleges, along with elements from progressive mainstream assessment thinking that can inform assessment in a tribal college setting. Chapter 5 overviews relevant approaches to methodology — including the case study method and the Context-Input-Process-Product (CIPP) M o d e l — and data analysis. Chapter 6 articulates an evaluation framework for the context, inputs, processes, and products o f N W I C s assessment program by outlining in each section a set o f elements or criteria that may be taken into consideration when establishing an assessment program in a tribal college and summarizing the results o f the research. In chapter 7,1 overview the responses to the three research questions and present a list o f elements that might assist tribal colleges to develop their assessment  11  programs. The thesis concludes with a discussion o f the theoretical and applied implications o f this research, the limitations o f this study, and suggestions for future research and work. One last note before I conclude this chapter: Because there is no one term by which Native Americans prefer to refer to themselves, in this doctoral thesis, the use o f terms such as tribal, Indian, Indigenous, Aboriginal, First Nations, and Native American reflect the vocabulary most appropriate to each context. Certain terms are used because they were used at the time by Native Americans, because they are preferred by one o f the primary audiences for this doctoral thesis (the Native American members o f the tribal college community), or because they are a part o f common names, such as tribal college, Bureau of Indian Affairs, etc.  12  C H A P T E R 2 NATIVE A M E R I C A N S IN H I G H E R E D U C A T I O N — HISTORY A N D O V E R V I E W O F TRIBAL C O L L E G E S  The tribal college system i n the United States is poorly understood and has not been recognized for what it has accomplished. The primary purpose o f this chapter is to outline the history, early development, and contemporary state o f tribal colleges. Tribal colleges are a relatively new phenomenon in the torturous and painful history o f education for Native Americans in the United States. Neither the significance o f this achievement nor the conceptual bases underlying the creation and functioning o f these colleges can be fully appreciated divorced from an understanding o f the historical experience from which they evolved. This chapter, therefore, reviews both that historical experience and the contemporary picture o f tribal colleges. The first section o f this chapter provides an overview o f the history o f Native Americans in U . S . higher education through three distinct periods: (1) the colonial period, when the objective was converting Native Americans to Christianity through Indian missions in early colleges; (2) the federal period, during which the federal government largely ignored Native American higher education, yet enacted a series o f policies that were devastating to the Indian community; and (3) the self-determination period, characterized by significant shifts in federal policy toward self-determination, as w e l l as the founding o f tribally controlled colleges (Carney, 1999). Each o f these periods significantly influenced the evolution and direction o f higher education for Native Americans in the United States. The second section o f this chapter examines the evolution and founding o f tribal colleges in the United States, beginning with Navajo Community College, the first tribal college i n the U . S . It provides an overview o f the most significant policies and events that led to the establishment o f tribal colleges, and the current state o f these colleges, including an examination of the unique features o f tribal colleges that increase Native American student access, retention,  13  and success. Finally, it summarizes other fundamental characteristics tribal colleges have in common, including shared challenges.  T h e H i s t o r y of N a t i v e A m e r i c a n s i n H i g h e r E d u c a t i o n i n the U n i t e d States  The Colonial Period (1500s - 1775) Traditionally, the education o f Indian youth consisted o f the family, extended family, and elders preparing them for the challenges they would face throughout life. Through this unstructured process, life skills, history, and religious and cultural norms and expectations were passed on (Oppelt, 1990). In contrast, the objective o f early European North Americans, from the arrival o f Columbus, was the " c i v i l i z i n g " o f Native Americans through conversion to Christianity and the elimination of their traditional ways (Oppelt, 1990). Conversion, they believed, would necessitate making Native Americans literate. Thus, during the early 1600s, higher education for Indians focused on literacy and vocational training combined with the primary goal o f converting them to Christianity (Carney, 1999). Throughout both the colonial and, later, the federal period, the practice o f removing Indians from their homes to be educated was a predominating feature o f White education for Indians. This practice likely originated in 1568, when the Jesuits established a school i n Havana, Cuba, to serve Indians from Florida (Carney, 1999). This school was the first attempt o f many i n the last 450 years to remove Indians from their homes, a practice that severed them from the cultural influences o f their families and tribes, with the underlying intention o f assimilating them into mainstream society. The first effort to establish a college for Indians, which occurred in the Jamestown settlement o f Virginia in 1616 to 1622 (Carney, 1999), was unsuccessful; but had it been  14  successful, it would have been the first college o f any kind in the United States. In 1616, Pocahontas, the first convert in Jamestown, traveled to England to raise funds for educating Indians. K i n g James was so impressed with her that he ordered the church to send money to fund a university for Native Americans in Jamestown, which he wanted to call Henrico College in honor o f his son. Unfortunately, the large sums o f money contributed for the college were diverted to the Virginia Company. In 1622, after an Indian uprising against the Virginia settlement that killed many colonists (Carney, 1999), the idea o f founding a college for Indians in Jamestown ended; however, the practice o f sending Indians to England to raise funds for Indian education, deceptive as it continued to be, lasted throughout the colonial period. Surprisingly, o f the first nine colleges established in the United States, three (Harvard, W i l l i a m and Mary, and Dartmouth) included Native Americans in their vision. In 1636, the N e w College (now Harvard University) in N e w England was the first college founded in the United States. In its 1650 charter, Harvard professed as one o f its goals "the education o f the English and Indian youth o f this country in knowledge; and godliness" (quoted in Carney, 1999, p. 1). Although the first Indian student attended Harvard in 1653, it was not until 1665 that the first one graduated. Few Indians ever attended Harvard until the 20th century, largely because o f its irrelevant curriculum and the rampant sickness and disease ravaging Indian communities (Carnegie, 1997). In 1970, Harvard experienced an increase i n Native American enrollment and renewed its commitment to Native American education by establishing the American Indian Program at the Graduate School o f Education. Between 1970 and 1995, Native American students in the Graduate School o f Education at Harvard earned 156 master's degrees and 19 doctorates (Encyclopedia o f North American Indians, 2005c); however, in 2003-2004, only 0.7% of the student body at Harvard was Native American (Mass Mentor, 2004), thus demonstrating  15  that Harvard still struggles to fulfill its original mission to educate Indians, among others (Karabel, 2005). In 1693, the College o f W i l l i a m and M a r y i n Virginia became the second college chartered by the crown i n the United States. Its mission statement expressed the desire "that the youth may be piously educated in good Letters and Manners, and that the Christian faith may be propagated amongst the Western Indians, to the Glory o f Almighty G o d " (quoted in Carney, 1999, p. 1). A handful o f Indian students attended the college between 1705 and 1721 and then, again, from 1743 to 1776. Most o f these students received support from the Boyle endowment, the last student receiving funds in 1776. In 1783, the Indian-serving section o f the school closed, and the endowment was reallocated to provide higher education opportunities for Black students (Oppelt, 1990). In 2003-2004, less than 0.5% o f the College o f W i l l i a m and M a r y ' s total enrollment were Native American and 5.5% were Black (Electronic Campus, 2004b). The next six colleges established i n the United States, founded between 1701 and 1766, had no Indian mission. These six colleges are now Yale University in Connecticut, Princeton University of N e w Jersey, Columbia University i n N e w Y o r k , the University o f Pennsylvania, B r o w n University i n Rhode Island, and Rutgers The State University o f N e w Jersey (Wikipedia, 2005). Although none o f these colleges encouraged Native Americans to enroll, the College o f N e w Jersey did admit three Indian students, none o f whom graduated during that time (Carney, 1999). Finally, in 1769, the ninth college, Dartmouth College in N e w Hampshire, also professed a mission for educating Indian students and included in its charter "the education and instruction o f youths o f the Indian tribes i n this Land in reading, wrighting, and all parts o f Learning which shall appear necessary and expedient for civilizing and Christianizing Children o f Pagans as well  16  as in all liberal Arts and Sciences; and also o f English Youth and any others" (quoted in Carney, 1999, p. 1). A s Pocahontas had done earlier, another Native American, Samson Occum, traveled to England in search o f funds for Indian higher education. Born i n 1723, Samson Occum, a Mohegan, was one o f the finest scholars o f his time and eventually became a teacher and a Presbyterian minister. He was revered as the perfect example o f a " c i v i l i z e d " and educated Christian Indian (Oppelt, 1990). H i s teacher, Eleazar Wheelock, was an evangelical minister. Thanks to Wheelock's success in educating Occum, Wheelock became excited about the idea o f educating Indians. Wheelock convinced Occum to travel to England and Scotland i n 1766 and 1768 to raise money for Indian higher education. The fund-raising trip was an immediate success, and, as a result, Dartmouth College was established in 1769. Although he was successful in raising funds that led to the establishment o f Dartmouth, Occum later felt betrayed when few Indian students ever attended the college and the funds he raised were not used for their intended purpose. Although Dartmouth demonstrated minimal success in educating Native Americans, its recent history has been more promising. With a new president i n 1969, Dartmouth was "refounded" (Encyclopedia o f North American Indians, 2005b). This president reaffirmed Dartmouth's commitment to Native American higher education and made a pledge to maintain 3% Native American undergraduate enrollment. He has also devoted significant financial assistance and resources to their education. The undergraduate Native American graduation rate has increased from 50% in 1970 to 72% i n 2004 (Dartmouth College, 2005) and, in 2003-2004, 3% o f Dartmouth's total undergraduate enrollment (i.e., from 4,098 students total) was Native American (Xap, 2004). In the first 125 years o f its existence, only 58 Indian students attended Dartmouth College, and 11 graduated (Carney, 1999).  17  During the period when these first U . S . colleges were founded, the purpose o f recruiting Indian students was essentially either to access funds devoted to an Indian mission or to convert Indian students to Christianity. The first three colleges to profess a mission for Indians all experienced similar financial challenges: fraudulent behaviour, misappropriation o f funds from Indian education to the general education o f the colony, and a small Indian population base from which to increase Indian student enrollment. The only source o f large donations for the colleges was England, where fundraisers were sent on the false pretext o f raising money to educate Indians (Carney, 1999). Whatever the intentions o f these colleges, 50 Native Americans actually attended these early institutions before 1776, and one graduated (Carney, 1999). A number o f factors may have contributed to these failed, half-hearted experiments in higher education for Indians. For example, many Indians resisted a White education, believing it would be irrelevant to their vocation and culture and fearing it would lead to the acculturation o f their people (Oppelt, 1990). The racism and lack o f support that Indian students experienced from the local colonists also hurt the missionaries' cause (Oppelt, 1990; Szasz, 1988). Furthermore, few people in society could attend college at that time, as a college education was reserved for only the elite male population. The colleges' presidents eventually realized that unless Native Americans found a practical and meaningful use for their education, higher education for Indians would remain a futile exercise (Carney, 1999).  The Federal Period (1775 - 1928) The so-called federal period in the history o f Native American education cannot be easily characterized, except that it was devastating to Native American communities and education. It was during this period that the new federal government was delegated responsibility for the education o f the Indian peoples under its jurisdiction, but this did not signal the beginning o f  18  either consistent or beneficial policies and approaches to Indian education. T w o competing, and equally dreadful, points o f view alternately dominated the direction that Indian education took for much o f this time: (a) the assimilationist view, which favored vocational training for Indian people to prepare them for manual labor in White society and (b) the "removal" view, which favored removing Indians from White society by relocating them to separate, segregated tracts o f land. The assimilationist view ultimately dominated (Carnegie, 1997). During both the colonial and the federal periods, the culture and values o f Indians attending colleges were neither appreciated nor reinforced. In addition, Indians had no control over the purpose o f their own education and thus feared they would lose their culture i f they attended these institutions (Carnegie, 1997).  Treaty Period (1778 - 1871) With the founding o f the United States i n 1776, the responsibility for Indian-related endeavours was delegated to the federal government (Carney, 1999), and Indian tribes now had to deal with this new authority. During the treaty period (1778 to 1871), the government negotiated 645 treaties with tribes (97 o f which included minimal educational provisions), thus recognizing the sovereignty o f those communities. Tribes therefore became "dependent nations" of the federal government (Carney, 1999). This began an era, lasting until the 1960s, when the higher education o f Indians was largely neglected.  The Indian Civilization A c t (1819) The Indian Civilization A c t o f 1819, the most important legislation during the federal period, granted $10,000 a year for educating Indians i n "civilized" agricultural practices and Christianity (Carney, 1999; Oppelt, 1990). In effect, only a small amount o f this money was  19  ever allocated to Indian education (University o f Illinois, 2000). Because the government had been unsuccessful in its attempts to educate Indian children up to this time, from 1770 to the 1870s, these minimal funds for educating Indians were channeled through missionaries who, once again, were in charge o f the Indians' education. The purpose o f this religious instruction was yet again mass conversion (Carnegie, 1997) — it also called for the obliteration o f Indian religious practices, dances, and cultural practices (University o f Illinois, 2000). Because it provided money to churches to educate and promote Christianity, the Indian Civilization A c t program ended in 1870 when controversy arose over the issue o f the separation of church and state (Carney, 1999). During the early 1820s, when the federal government's support for Indian education consisted o f minimal financial assistance for Indian students attending eastern colleges, there were no higher education institutions for Native Americans i n the U . S . (Carney, 1999). It was also during this time that the historical trend o f the government's not allocating appropriated funds for Indian education originated, as the minimal provisions for Indian students were infrequently honored (Carney, 1999).  Removal Policies (1825) In 1825, the forced relocation o f Indians westward, with its concomitant loss o f their lands, became the primary federal policy for Indian populations (Carney, 1999). The little involvement the federal government had with education during this period was limited to policy and funding. Educational policy shifted its focus from higher education (to convert and civilize Indians) to vocational training (combined with religious education) to prepare Indians for largely manual jobs.  20  About this time, several tribes recognized that getting a White education might help their cause (Oppelt, 1990). Most notably, the Cherokee and Choctaw tried to negotiate provisions for education in many o f their treaties (Carney, 1999). The Choctaw even planned a higher education institution, but their plans fell through when they were forcibly removed from their lands. Nonetheless, the Cherokee and Choctaw were still able to obtain funding for some o f their top students to attend colleges in the east (Barbara Wright & Tierney, 1991). Later, after the 1  C i v i l War (1861 - 1865), the Choctaw were able to provide scholarships from tribal funds for students to attend college. Both tribes established excellent school systems and remained committed to higher education (Carney, 1999; Oppelt, 1990).  Assimilation Policies (1870s) Since their first encounter with Indians, Whites have discussed the pros and cons o f assimilation versus removal and have been rather haphazard in their attempts at both. U p until the 1860s, removal advocates predominated as Indian tribes were forced farther and farther west (Carney, 1999). In the 1870s, however, assimilation replaced removal as the dominant policy. A s White populations continued to increase, there was less available land on which to relocate Indian communities. In 1870, the federal government provided $100,000 for Indian industrial schools that were to be under federal control for the first time (Barbara Wright & Tierney, 1991). The underlying intentions were again to inculcate White values and to assimilate Indians into lowlevel positions in society. Furthermore, because individual Indians would soon be allocated  1  Mainly from the Civilization Fund and Scottish (Occum's) Fund.  21  tracts o f land, it was thought that Indians should be educated about farming practices (Carney, 1999). Federal assimilation policy found its expression in two significant pieces o f new legislation: the Dawes General Allotment A c t (1887) and the Curtis A c t (1898). Together, these two acts affected two essential aspects o f the Indian way o f life and values: education and land (Szasz, 1999). The Dawes General Allotment A c t officially removed land from tribal control and allotted individuals parcels o f land. The logic was that it would be easier to assimilate Indians into the larger society i f there were no tribal structures or tribal land (DeJong, 1993). The Curtis A c t abolished tribal courts and governments (DeJong, 1993) and resulted in the loss of tribal sovereignty (Carney, 1999). Throughout the 1870s, the government's approach to Indian education also included moving students from their families and tribes into residential schools, whereby, it was believed, they could be assimilated into American society (Carnegie, 1997). In 1879, the first government boarding school was established i n Carlisle, Pennsylvania. The Carlisle Indian School was founded by Richard Pratt and, by 1899, 25 "Carlisle" schools had been established, with 20,000 Indian students attending 25 boarding schools in 15 states (Encyclopedia o f North American Indians, 2005a; Szasz, 1999). In these schools, Native American children were provided with vocational education and some religious training. It was hoped the children would later return to their families as missionaries and convert them to Christianity (Carney, 1999). Children were forced to speak English and were not allowed to speak their Native languages or practice their culture in any way. Pratt honestly believed Native American students could excel in that environment and that they had a capacity to learn, a belief held by few people at that time — he also believed that it was best to " k i l l the Indian and save the man" (Encyclopedia o f North  22  American Indians, 2005a, p. 1; Szasz, 1999). M a n y o f the children were physically, emotionally, and sexually abused — and hundreds o f them died under these harsh circumstances (Encyclopedia o f North American Indians, 2005a; Szasz, 1999). The approach was deeply flawed: Dropout rates were high, attendance was not mandatory, inappropriate training was provided, and no employment was offered to graduates in the community. Thus, in 1917, the last Carlisle school closed and Indian students returned to their communities and their traditional ways o f life (Carnegie, 1997). A s the schools failed, so too did the assimilation policy. Despite the failure and closing o f the Carlisle schools, Pratt was able to convince the public that Indians could be educated. A s a result, these schools were the progenitors o f what finally evolved into more progressive educational initiatives. Some o f these schools developed into vocational boarding schools, a number o f which have survived and operate today (Szasz, 1999). Haskell Indian Nations University in Lawrence, Kansas, was established i n 1884, and the Institute o f American Indian Arts in Santa Fe, N e w Mexico, was founded in 1890 (Szasz, 1999). Both o f these institutions are now federally chartered and provide exclusively Native American higher education opportunities. In particular, Haskell has been influential in that it has provided, and it continues to provide, Native Americans from diverse backgrounds with culturally relevant educational programs. Thus, Pratt's vision and his belief in educating Indians — however misguided it was — considerably shaped the progress and course o f post-secondary education for Native Americans in the United States.  The Evolution o f Private Indian Colleges (1880s) Only two privately funded Native American colleges originated in the 19th century: "Indian University" (now Bacone College) in Muskogee, Oklahoma, and the Croatan Normal  23  School (now the University o f North Carolina at Pembroke) (Carney, 1999). The first private Native American college, Bacone College (established in 1880), was based on Christian principles. Although it was also open to non-Natives, it was the first tribally controlled college and the only one in the U . S . for many years (Carney, 1999). A l m o n Bacone was a missionary and teacher, a proponent o f providing a curriculum built upon the local culture and taught in the communities themselves. He was concerned about the decreasing Indian population. The Creek Council granted the college 160 acres to establish a campus i n 1910. Bacone College was the first land grant college founded by Native Americans (Barbara Wright & Tierney, 1991). In 2003 to 2004, 39% o f its 914 enrolled undergraduate students were Native American (Electronic Campus, 2004a). The second private Native American college, the University o f North Carolina at Pembroke, was founded in 1887 and is the only state-supported four-year college founded specifically for Native Americans (Carney, 1999). Initially, the General Assembly o f North Carolina established it to serve the Lumbees (i.e., the Croatan), with the aim o f educating Indian teachers. Over the years, the college struggled financially and finally evolved into a mainly White institution. Although from 2003 to 2004, 2 1 % o f all enrolled undergraduate students at the University o f North Carolina at Pembroke were Native American (of a total o f 3,813), the college has abandoned its focus on Native American culture (Carney, 1999; Electronic Campus, 2004c). Between 1887 and 1954, Bacone College and the University o f North Carolina at Pembroke were the only private institutions o f higher education in the United States to serve mostly Native American students. It was difficult to establish Native American colleges for a number o f reasons, including pervasive racism, the small Native American population base, the  24  dispersion o f Native Americans, and federal control o f all levels o f Indian education (Carney, 1999). Furthermore, the only government assistance available to Native Americans in higher education at that time was limited scholarships and loans (Carney, 1999). In addition to Bacone College, there are two other small private Native American colleges still functioning today: American Indian College in Phoenix, Arizona, which was founded in 1972, and Nazarene Indian Bible College in Albuquerque, N e w Mexico, which was established in 1975 (Albuquerque Colleges and Universities, 2005; American Indian College, 2005). American Indian College is sponsored by the General Council o f the Assemblies o f G o d and offers bachelor's degrees in Christian ministry and elementary education. Nazarene Indian Bible College offers limited bachelor's and associate's degrees in Biblical studies and human resources management.  The 2 0 Century th  During the late 1800s and early 1900s, changes in the perceptions and attitudes o f a number o f government officials, as well as the general public, toward Native Americans began to appear. More people were beginning to understand not only that Native Americans were worthy of a higher education but also that their education would benefit all o f society. Furthermore, they were starting to believe that acculturation did not benefit society, that Native American culture should be valued and preserved (Carney, 1999). Despite these changes, Native American higher education, in practice, remained substandard and continued to offer only a basic curriculum that focused on low-level vocational skills (Carney, 1999). B y the turn o f the century, the Native American population in the United States was less than 250,000, the lowest it had ever been, compared to an estimated 5 million at the time o f  25  Columbus's arrival (Carnegie, 1997). Tribal governments had been powerless to stop the 2  federal government's attempts to eradicate their cultural identity and sense o f self (Carney, 1999). B y this time, the federal government's policies had had a devastating impact on Native American communities across the country, which had lost much o f their language, land, culture, and hope. Disenfranchised in every way, Native American communities had become dependent on the government for their survival (Carnegie, 1997). In the early 1900s, only some tribes, one state, and some religious organizations paid any attention to Native American higher education; the federal government essentially ignored their plight. Since the early 1900s, some individuals believed that Native Americans should control their own higher education institutions (Stein, 2003). In 1911, August Breuninger made the first serious proposal to start a university that would enable Native Americans better access to higher education (Crum, 1989). He envisioned a curriculum that would focus on Native American culture and proposed that the college be associated with a museum. Breuninger's ideas, as well as others', were dismissed by assimilationists, and, over the next 50 years, other proposals to start Native American vocational colleges also failed (Carnegie, 1997; Stein, 2003). In the late 19 and early 2 0 century, a trend began whereby schools evolved into normal th  th  schools, which trained teachers, and then, occasionally, into colleges. Instead o f transforming into Indian colleges, however, a number o f Indian schools instead became White colleges, because o f the financial need for a wider student base (Carney, 1999). These included Ottawa University in Kansas, Sheldon Jackson College in Alaska, the University o f Tulsa in Oklahoma, Northeastern State University in Oklahoma (where Native Americans still have a significant  This number has been steadily increasing. In 2000, the U.S. Census reported that 4.1 million people (or 1.5% of the total population) claim some Native American ancestry (U.S. Census Bureau, 2004). 2  26  presence today), and Fort Lewis College in Colorado (Carney, 1999). The Indian students from these closed schools were sent to the public schools, where their culture was unappreciated (Carney, 1999). B y 1920, interest in Native American culture increased significantly in intellectual circles. Reformers became interested in the Pueblo Indian society and the preservation o f Native American culture (Carney, 1999). This movement was to set the stage for the next era.  The Self-Determination Period (1928 - present) U p until this time, "the principle of local citizen control o f schools, that is a keystone o f public education in the United States, was never extended to the education o f Indians because they were deemed to be unqualified to manage their own schools" (Oppelt, 1990, p. 14). Things were about to change as the 2 0 century brought with it the most significant federal policy shifts th  toward self-determination in history (Carney, 1999). Two major reforms forced the government to begin to relate differently to Native Americans. The first, the Snyder A c t o f 1921, gave the Bureau o f Indian Affairs direct responsibility for Indian higher education, even though, as in previous years, this was still limited to the support o f students rather than institutions (Carney, 1999). The second, passed in 1924, was the Indian Citizenship A c t , which granted U . S . citizenship to all Native Americans (Carney, 1999). These two pieces o f legislation were significant in that they forced the government to regard Native Americans more justly, as citizens who deserved the right to control the future o f higher education. It also set the stage, a number o f years later, for the dreadful findings o f the Meriam Report.  27  The Meriam Report (1928) In 1928, the Meriam Report, which was also known as the "Problem o f Indian Administration," condemned the federal government for the abuse Native Americans had long endured (Carnegie, 1997; National Advisory Council on Indian Education, 1993). It particularly criticized federally run boarding schools, charging that because o f the abuse experienced in these schools, the situation for Native American communities was bleak. It criticized the emphasis on vocational training, finding it completely unrelated to Indians' traditional ways and providing little hope for employment. It found that Native Americans were prevented from managing their own programs and being forced to use substandard services (i.e., health and education programs). The report encouraged more Native Americans to pursue higher education and suggested the federal government provide additional funding and financial aid for this purpose (Oppelt, 1990). Although the report recognized the importance o f traditional culture and values, as well as community-based education, it stopped short o f recommending the establishment o f tribally controlled colleges (Carney, 1999). Nonetheless, the government's gradual shift from its paternalism to acceptance o f self-determination stemmed from this report. In its wake, many boarding schools were closed, more Indian high schools with a greater percentage o f Native American teachers were established, Native Americans were trained to serve in Bureau o f Indian Affairs positions, and Native American culture was formally and actively preserved through financial support. These were the first concrete steps the federal government took away from the policy o f acculturation. In 1932, as a result o f the findings o f the M e r i a m Report, President Franklin Roosevelt and Commissioner o f Indian Affairs, John Collier, initiated significant changes affecting Native American communities (Carnegie, 1997). Collier, who was knowledgeable about Native American cultures and innovative community-based education models, began making decisions  28  based on Native American needs, advocating self-government and sovereignty for Native American communities (Oppelt, 1990). The goal now became empowering Native communities to restore their culture and to work with the government. For example, because most teachers were White, there was a movement to train Native Americans teachers. The situation was improving, but White men still continued to make most policies, and Native Americans themselves had little voice in the design o f their own education. In addition, the potential for improved opportunities was set back when funding for these programs was reduced during and after W o r l d War II (Carnegie, 1997). Despite these challenges, the government's attitude toward Native Americans had changed significantly, paving the way for Indian self-government.  The Indian Reorganization A c t (1934) With the advent o f the Indian Reorganization A c t (also known as the Wheeler-Howard A c t or the Indian B i l l o f Rights) in 1934 and the concomitant termination o f the Dawes General Allotment A c t , tribal communities were finally given the right o f self-government (Oppelt, 1990). Thus, Native American history moved from a dark period o f assimilation and elimination to a period o f tolerance and preservation (Carney, 1999). The Indian Reorganization A c t resulted in the creation o f a minimal loan fund for Native American college students and increased funding for higher education (Oppelt, 1990). A t about the same time, the Johnson-O'Malley A c t also provided substantial vocational student loan funds and allowed states to provide education. Nonetheless, these two acts had little effect on increasing the numbers o f Native Americans taking advantage o f the opportunities for higher education because the education offered was still not relevant to Native American students, nor did it recognize or support their culture and values (Carney, 1999).  29  During the late 1940s and early 1950s, there was little apparent progress. The Bureau o f Indian Affairs, and some tribes, did begin scholarship programs, but the Bureau o f Indian 3  Affairs primarily concentrated its efforts on establishing elementary schools, a number of high schools, and a couple o f technical institutes. A s a result, no tribally controlled colleges were formed during this time. The success o f the scholarship programs was negligible as well; students still faced financial limitations, cultural barriers, inadequate academic preparation, minimal student support services, and long distances to the colleges (Carney, 1999). B y 1954, Congress again terminated its responsibility for Native Americans and thus funding for more than 100 Native American tribes (Carney, 1999).  "Indian Education" This discussion would not be complete without pointing out that i n Native American communities, there has always been disagreement over the appropriateness o f Native Americans pursuing higher education. The juxtaposition o f the two words 'Indian' and 'education' has almost always been problematic in spite o f the agreement by Indian parents and Anglo policymakers on the importance o f education for Indians (Bradley, 1980). Part o f the problem is that Indian education is inherently a bicultural enterprise that has been directed at two sometimes competing and sometimes complementary goals: assimilation and self-determination (Havighurst, 1981). (Hampton, 1995b, pp. 7-8)  The purpose o f "Indian education" is to know oneself; to determine from where one has come, as well as one's passions (vocation); and to dedicate one's life to those pursuits. The ultimate goal is to become a complete individual and to provide the next generation with a sense o f what is important and what is valued in society (Cajete, 2000). "Indian education must enhance  3  The Bureau of Indian Affairs scholarship program started in 1948.  30  Aboriginal consciousness o f what it means to be an Indian, thus empowering and enriching individual and collective lives" (Battiste, 1995, p. xv). In order to do this, knowledge o f one's community is essential (Marker, 2004). Finally, "the effectiveness o f education is measured by how well it prepares people to handle the problems and opportunities o f life i n their own time and place" (Watt-Cloutier, 2000, p. 114). For over 350 years, the few Native Americans who could pursue higher education had to attend mainstream colleges that aimed at assimilating them into White society or recruiting them for missionary work (Carnegie, 1997; Karabel, 2005). These attempts to "educate" the Indians largely failed. Finally, by the late 1950s, the c i v i l rights era that was on the horizon, combined with the new community college movement, would bring new momentum and attitudes to support the founding o f the first tribally controlled colleges (Stein, 1992). These efforts would be the first attempts o f pursuing the vision o f a true Indian education.  T h e F o u n d i n g o f T r i b a l Colleges i n the U n i t e d States (1960s)  In the early 1960s, Forbes, a Native American professor i n California, proposed creating a Native American university. H i s vision for the university included providing kindergarten to grade 12 teacher training; recording the written form o f Native American languages; teaching language, culture, and arts; and offering adult basic education courses to bring adults up to college-level work. Most importantly, he believed that Native Americans should control their own institutions (Stein, 2003). Forbes laid the groundwork for the tribally controlled colleges that were established i n the following decade. Deganawidah-Quetzalcoatl University, founded i n 1971 in Davis, California, was based on Forbes's proposal (In Search o f Aztlan, 2004). A l s o during the 1960s, greater numbers o f Native Americans started attending college (Oppelt, 1990). This trend coincided with a number o f factors and events. First, Native  31  American leaders renewed their commitment to self-determination, and tribal members increasingly regarded higher education as a way to serve the tribe, advance the process o f selfdetermination, and restore traditional culture and knowledge (Carnegie, 1997). Second, in 1968, under President Johnson, the government committed to the policy o f "self-determination" (Carney, 1999). Every president since that time has supported this policy (Carnegie, 1997). Third, i n 1966, for the second time ever, a Native American, Robert Bennett, was named commissioner o f the Bureau o f Indian Affairs.  4  He was committed to self-determination and,  thus, education became a priority (Carney, 1999). Finally, following W o r l d War II, access to higher education improved when the G I B i l l provided funding for returning veterans to pursue college degrees (Carnegie, 1997). M a n y Native American veterans took advantage o f this opportunity but, discouraged by racism and discrimination, also dropped out at unusually high rates (Carnegie, 1997).  Navajo Community College (1960s) In 1957, the Navajo Nation, with a population near 200,000, realized that with few members having the ability to lead the tribe, they needed to provide higher education i f they were to attain self-determination (Carnegie, 1997). To achieve this objective, the Navajo Nation established a scholarship fund to enable tribal members to attend colleges and universities. Over 50% o f the students awarded scholarships dropped out by the end o f their first year because they could not navigate the foreign cultures o f those institutions (Carnegie, 1997; R. Nichols & Monette, 2003). Then, in 1966, the federal government funded educational services to the Navajo tribe through the Rough Rock Demonstration School. This successful project eventually  4  The first Native American was named commissioner of the Bureau of Indian Affairs in 1869.  32  led to the establishment o f Navajo Community College (now called Dine College) i n 1968, the first tribally controlled college in the United States (National Advisory Council on Indian Education, 1993). The college was federally funded and secured further funding from the Navajo tribe, the Office o f Economic Opportunity, and the Donner Foundation. The clearly articulated principles implemented at Navajo Community College became the model for all future tribally controlled colleges: The college would be controlled and managed by the local people " i n order to enhance the understanding o f their heritage, language, history, and culture" (Stein, 2003, p. 31); would foster a sense o f cultural pride and self-esteem, by integrating tribal culture, language, art, and history; would encourage the ability o f students to work effectively in the larger society; would provide individualized attention to address social as well as academic challenges; and would seek new innovative teaching methodologies to effectively transmit knowledge (Carnegie, 1997; Stein, 2003). Coinciding with the c i v i l rights movement, the tribal college movement arose from an increased sense o f pride and a resolve to legitimize and preserve cultural traditions, values, and knowledge through the process o f self-determination (Carney, 1999). These objectives were to be accomplished through the creation o f higher education institutions that would provide some o f the many needed academic and vocational skills to tribal members. Some tribes experimented by bringing local community colleges into their communities, but, because the curriculum was culturally irrelevant, the effort died out (Carnegie, 1997). The closest higher education model available at the time was that o f the rapidly expanding community college system (Stein, 1992). Therefore, founders o f the tribal college movement chose to base the tribal college system on that model, which embraced community control (Stein, Shanley, & Sanchez, 2003). Tribal colleges and community colleges both valued  33  the importance o f local control, local needs, open admission, job training, and local community development. One difference was that tribal colleges had as their goal the self-determination o f Native people through education. Other differences between community colleges and tribal colleges lay primarily in the sources o f funding, the governance structure, and the focus on culture (Carnegie, 1997; Stein, 1992). Following the example o f the Navajo, in the 1970s, several other tribes founded their own tribally controlled colleges. In 1970, Deganawidah-Quetzalcoatl University in California was established, followed, i n 1971, by Sinte Gleska University and Oglala Lakota College, both i n South Dakota (Stein, 1992). During their accreditation processes, many tribal colleges maintained relationships with mainstream universities and colleges (e.g., the mainstream institution would provide courses through the tribal college). Indian leaders reasoned that agreements made with non-Native American colleges and universities would provide support while the tribal colleges were gaining strength in their pursuit o f self-determination (R. Nichols & Monette, 2003).  Federally Supported Native American Colleges (1962 - 1971) The Bureau o f Indian Affairs currently operates and funds two Native American higher education universities: Haskell Indian Nations University i n Lawrence, Kansas, which evolved from an Indian school in 1884 and became a junior college in 1970, and Southwest Indian Polytechnic Institute, in Albuquerque, N e w Mexico, which also evolved from an Indian school and then was founded in 1971 as a higher education institution providing technological training (Carney, 1999; Crazy B u l l , 2007). The Institute o f American Indian A r t in Santa Fe, N e w Mexico, is a federally chartered institution with a specific mission to support Native American arts and culture. It was founded i n 1962.  34  The Kennedy Report (1969) and Indian Education A c t (1972) In 1969, about mid-way through the period when the Bureau o f Indian Affairs established its first tribal colleges, the Kennedy Report, also known as the report o f the Senate Special Subcommittee on Indian Education (chaired by Robert Kennedy and then later by Edward Kennedy), examined Indian education and the dismal rate o f success o f students at all levels (DeJong, 1993). Reminiscent o f the M e r i a m Report, there was still no mention o f the importance o f establishing tribally controlled colleges. The years following the release o f this report saw a flurry o f bills supporting Indian education. One o f these, the Indian Education A c t of 1972, required Indian education programs across the country to address low achievement and high dropout rates among their students. Another, the Indian Self-Determination and Educational Assistance A c t o f 1975, committed Congress to improving the social and economic conditions o f Native Americans significantly by providing funding for technical training, Bureau of Indian Affairs support, and financial assistance for taking control o f their own programs (Carnegie, 1997). It also gave tribes the right to contract with the federal government to operate educational programs serving their tribal members, such as Head Start, while maintaining local control o f elementary and high schools. This added control over educational programs provided Native Americans with the impetus for creating additional educational opportunities i n their communities and the confidence that self-determination was more than a hope.  The American Indian Higher Education Consortium (1973) In 1973, the presidents o f the first six tribal colleges founded the American Indian Higher Education Consortium ( A I H E C ) , an institution that represents the interests o f the tribal colleges and is instrumental i n supporting their work in assisting Native American communities i n their national effort toward self-determination (Benham, 2003). B y pooling their resources, A I H E C  35  assisted the early tribal colleges with accreditation procedures, curricula development, and planning (R. Nichols & Monette, 2003). Today, A I H E C represents 32 colleges in the United States and is governed by all the member institutions. Its mission is to maintain commonly held standards o f quality in American Indian education; support the development o f new tribally controlled colleges; promote and assist i n the development o f legislation to support American Indian higher education; encourage greater participation by American Indians i n the development o f higher education policy. ( A I H E C , 2004, p. 1)  The second central organization o f tribal colleges, the American Indian College Fund, was established in 1989 through A I H E C to raise funds and provide financial assistance to tribal colleges and their students (Stein et al., 2003). In 1986, tribal college students founded the A I H E C Student Congress "to promote leadership, self-governance, cultural preservation, educational achievement, community development, economic sustainability and success for all Native people" ( A I H E C Student Congress, 2004, p. 1).  The Tribally Controlled Community College Assistance A c t (1978) Three pieces o f legislation addressing Native education followed upon each other in 1978:  the Education Amendments A c t o f 1978, which focused on elementary and high school  education; the Higher Education Act, which provided increased funding both for Native American students attending college and for Native American college programs (Barbara Wright & Tierney, 1991); and, finally and most significantly, the Tribally Controlled Community College Assistance Act, which provided minimal federal funding for the main operations o f  36  tribally controlled colleges (Benham, 2003; Carnegie, 1997; R. Nichols & Monette, 2003). "After a lengthy struggle" (Shanley, 2003, p. 63) with A I H E C , the government provided this minimal funding annually to each tribally controlled college for each Native American student it served. Native Americans with new skills were required to implement this new legislation effectively; therefore, the need for educated leaders and higher education, specifically tribal colleges, became obvious (Carnegie, 1997).  Land Grant Status (1994) In 1994, through the M o r r i l l A c t , land grant status was granted to 29 tribal colleges (Stein et al., 2003). This important designation provided these colleges with support from mainstream institutions and provided minimal funding for agricultural and natural resource programs. Furthermore, in 1996, the White House Initiative on Tribal Colleges and Universities created a partnership between the federal government and tribal colleges and mandated that tribal colleges receive increased resources and services (Stein et al., 2003). From the beginning o f the self-determination period through 1994, measures were created with the intention o f improving Native American access to higher education, strengthening educational programs, increasing graduation rates, and educating people "who are grounded i n their own culture, yet are prepared with the social, civic, and work skills they need to live and contribute to a multicultural, global society" (Kellogg Foundation, 1998, p. 2). Tribal colleges were attempting to fill a void by providing a link between higher education i n mainstream society and tribal communities, so that their graduates would be capable o f functioning in both worlds (Carnegie, 1997).  37  Tribal Colleges Today  Purpose and Philosophy In the United States, tribal colleges evolved because existing systems o f higher education still required assimilation and were not benefiting Native American students, who were often geographically isolated, had fewer opportunities to access post-secondary education, or did not have the academic preparation to succeed i n mainstream post-secondary institutions. For example, i n 1984, only 3% o f Native Americans graduated from college, compared to 16% o f non-Native Americans (Tierney, 1991). A s Boyer (1997) notes, especially since [enactment of] the G I B i l l and the Higher Education A c t , the federal government, as well as individual colleges and universities had encouraged Indian students to enroll. But as more did, it became clear that access did not guarantee academic success. The dropout rate for American Indians remained at 90 percent or higher at many institutions, (p. 25)  In response to these concerns, tribal colleges were established for a two-fold purpose: (1) to preserve and restore traditional tribal culture and knowledge and (2) to provide certificates or associate's (two-year) degrees that would provide Native American students with better employment opportunities and facilitate transfer to four-year colleges or universities (Farm, 2002; Stein, 2003). Tribal colleges, in the broadest sense, are committed to the selfdetermination and survival o f Native peoples through education (Boyer, 2003). A s N W I C President Crazy B u l l (2004) elaborates, "self-determination means choice, access, and control over human and natural resources" (p. 1). The colleges provide Native American students from isolated areas, who might not otherwise pursue post-secondary options, with access to higher education (Boyer, 2003). In addition, as Boyer (1990) points out, more Native American students are attending tribal colleges to avoid racist and non-relevant curricula, to counter prior  38  negative experiences in mainstream community colleges and universities, and to be educated in an environment that addresses the needs o f tribal communities. In addition to serving individual students, tribal colleges also have a wider mandate: to address issues o f social justice as well as the social and economic needs o f their tribal communities, through research, instruction, service, and other innovative means ( A I H E C , 1998; Benham, 2003; Stein, 2003). Furthermore, tribal colleges provide a supportive learning environment for students who generally have not been successful i n mainstream colleges; promote tribal culture, art, history, traditions, and language (the preservation o f language is seen as being o f primary value in protecting Native American culture); provide services that enhance the communities they are in; and are centers o f research and scholarship that also benefit their tribal communities. Furthermore, tribal colleges provide Native American people with a means o f realizing equality and sharing i n the opportunities o f the larger society; for collective social and economic mobility; o f overcoming dependency and "neocolonialism"; o f providing the expertise and leadership needed by First Nations communities; to demystify mainstream culture and learn the politics and history o f racial discrimination.. .to address communal need for "capacity-building" to advance themselves as a distinct and self-determining society, not just as individuals. (Kirkness & Barnhardt, 1991, p. 4) Finally, in the context o f Native American perspectives and culture, tribal college programs also include some o f the more mainstream liberal arts purposes to which Axelrod (2002) refers, such as developing the critical thinking, analytical, and communication skills o f their students. Thus, tribal colleges provide access to post-secondary education, a first step to acquiring social capital (Bourdieu, 1986), and an essential step to self-determination.  Enrollment and Locations Since the establishment o f the first tribal college i n 1968, Native American participation in higher education has increased at a rapid rate, as more than one third o f all Native American  39  students in two-year colleges are attending tribal colleges ( A I H E C , 1998; Ortiz & HeavyRunner, 2003). Currently, there are 32 tribal colleges serving approximately 30,000 full- and part-time students, about 85% o f whom live below the poverty line ( A I H E C , 2005; Benham, 2003). A s indicated by Figure 2.1, tribal colleges are located throughout the United States, mainly in the northern plains states (i.e., Montana, North Dakota, and South Dakota).  40  Figure 2.1 Map of Tribal Colleges and Universities in the United States  Note. M a p from the American Indian College Fund (2007). Founding dates are in parentheses. Arizona  Montana  North Dakota  1.  Dine College Tsaile, Arizona (1968)  10.  Blackfeet Community College Browning, Montana (1979)  22.  Cankdeska Cikana Comm. College Fort Totten, North Dakota (1974)  2.  Tohono O'odham Comm. College Sells, Arizona (1998)  11.  Chief Dull Knife College Lame Deer, Montana (1975)  23.  Fort Berthold Community College New Town, North Dakota (1988)  12.  Fort Belknap College Harlem, Montana (1984)  24.  Sitting Bull College Fort Yates, North Dakota (1986)  13.  Fort Peck Community College Poplar, Montana (1978)  25.  Turtle Mountain Community College Belcourt, North Dakota (1972)  14.  Little B i g Horn College Crow Agency, Montana (1980)  26.  United Tribes Technical College Bismarck, North Dakota (1987)  15.  Salish Kootenai College Pablo, Montana (1977)  Colorado Amer. Indian College Fund (headquarters) Denver, Colorado (1989) Kansas 3  Haskell Indian Nations University* Lawrence, Kansas (1970)  Michigan 4.  Bay Mills Community College Brimley, Michigan (1984)  16.  5.  Keweenaw Bay Ojibwa Comm. Coll. Baraga, Michigan (1975)  Nebraska  6.  Saginaw Chippewa Tribal College Mount Pleasant, Michigan (1998)  17.  Little Priest Tribal College Winnebago, Nebraska (1996)  18.  Nebraska Indian Comm. College Macy, Nebraska (1978)  Minnesota 7.  Fond du Lac Tribal and Comm. College Cloquet, Minnesota (1987)  8.  Leech Lake Tribal College Cass Lake, Minnesota (1992)  9.  White Earth Tribal and Comm. College Mahnomen, Minnesota  * These colleges are federally chartered.  Stone Child College Box Elder, Montana (1984)  New Mexico  South Dakota 27.  Oglala Lakota College Kyle, South Dakota (1971)  28.  Sinte Gleska University Mission, South Dakota (1970)  29.  Sisseton Wahpeton College Sisseton, South Dakota (1984)  Washington 30.  Northwest Indian College Bellingham, Washington (1983)  19.  Navajo Technical College Crownpoint, New Mexico (1993)  20.  Institute of American Indian Arts* Santa Fe, New Mexico (1962)  31.  College of Menominee Nation Keshena, Wisconsin (1993)  21.  Southwestern Indian Polytechnic Institute* Albuquerque, New Mexico (1971)  32.  Lac Courte Oreilles Ojibwa Comm. Coll. Hayward, Wisconsin (1982)  Wisconsin  41  Although the largest tribal college (Dine College) has approximately 1,500 students, most tribal colleges are small, with enrollments o f less than 400. Even with such small enrollments, the colleges have the potential to significantly influence their communities where the populations they serve are usually also small (i.e., a couple o f thousand people) and graduating a handful o f tribal members can make a difference. Furthermore, because some tribal colleges are gaining reputations as "legitimate institutions o f higher learning" (Carnegie, 1997, p. 36), they now recruit Native Americans from outside their immediate areas.  Programs Initially, all 32 tribal colleges were established with the intention o f providing vocational training to upgrade tribal members' skills to become employed, and the majority still focus much of their curricular efforts in this direction through certificates and two-year degrees (e.g., construction trades, medical coding, office professional training) (Carnegie, 1997). Furthermore, though most began as two-year colleges, currently, seven tribal colleges offer four-year bachelor's degrees, and two offer master's degrees (American Indian Higher Education 5  6  Consortium, 2005). In addition, some two-year tribal colleges offer transfer degrees by forming partnerships and articulation agreements with local universities that provide opportunities for tribal college students to transfer and complete their bachelor's degrees (Carnegie, 1997). The most common programs o f study are the ones that directly support the social and economic development o f the tribes they serve: education (especially early childhood education and K - 8 teacher training), health, business, and environmental studies.  Sinte Gleska University, Oglala Lakota College, Salish Kootenai College, Haskell Indian Nations University, Institute of American Indian Arts, Sitting Bull College, and Turtle Mountain Community College. 5  6  Sinte Gleska University and Oglala Lakota College.  42  Students Overall, the students at most tribal colleges are different in significant ways from their non-Native American counterparts at other colleges (Boyer, 1990). Most obviously, approximately 80% to 85% o f students attending tribal colleges are Native American, and the average student is a single mother (typically i n her late 20s or early 30s) (Carnegie, 1997; Farm, 2002; Ortiz & HeavyRunner, 2003). Although the majority o f tribal college students work, most still have substantial financial limitations that affect their ability to provide themselves and their families with stable housing and food. College is a significant financial burden for most tribal college students. In addition, students often lack basic reading, writing, and math skills. A l l these factors combine to result in higher stop-out and dropout rates compared to students i n 1  mainstream community colleges (Ness, 2002; Oppelt, 1990). O n the bright side, most tribal college students are the first i n their families to attend college, approximately 40% o f the graduates pursue further education, and many graduates find meaningful work ( A I H E C , 2004; Farm, 2002; Bobby Wright & Weasel Head, 1990).  Factors Influencing Access, Retention, and Success i n Mainstream Higher Education Native Americans have been proportionately under-represented in colleges and universities since the advent o f higher education i n the United States (Kirkness & Barnhardt, 1991). Native American students face two major hurdles related to college: getting accepted into and graduating from a mainstream college (Carnegie, 1997). Bourdieu's (1986) work on class reproduction clearly demonstrates that many people do not have a real "choice" to pursue their education further because o f social-structural barriers that prevent them from advancing. Rather  7  A student stops-out when he or she skips a term and returns at a later time.  43  than assist individuals to advance, education systems perpetuate social inequalities. Life experiences and employment outcomes can still be predicted based on social class and gender, demonstrating that all people do not have equal opportunities to learn (Andres, 1994; Dougherty, 1987; Furlong & Cartmel, 1997; Guiton & Oakes, 1995; M c C a l l , 1992). Bourdieu (1986) regards capital as "accumulated labor (in its materialized form or its "incorporated," embodied form) which, when appropriated on a private, i.e., exclusive, basis by agents or groups o f agents, enables them to appropriate social energy i n the form o f reified or living labor" (p. 241). Capital is "the set o f actually usable resources and powers" (Bourdieu, 1984, p. 114) and it takes time to amass. Social and cultural capital relate most to this thesis. He defines social capital, or power derived by social means, as the connections an individual has to social networks and resources, including to people and groups (1986). Social capital plays an important role in the acquisition o f other forms o f valued capital. Cultural capital is the accumulation o f advantages a person has as a result o f cultural resources, which are passed on primarily through the family, such as behaviour, attitudes, and habits. The various forms o f capital an individual accrues as a result o f life experiences w i l l assist h i m or her to realize his or her goals. The course o f an individual's life is a product o f interacting factors related to social structure, individual agency, organizational processes, and historical precedence (Gerhardt, 1996). In addition to economic barriers, a number o f social and cultural barriers self-exclude Native Americans and make it difficult for them to pursue higher education (Anderson, 1993; Ortiz & HeavyRunner, 2003). The three factors most commonly identified i n the literature that influence the access, retention, and success o f Native American students in higher education and  44  are related to cultural and social capital are (a) family support and background, (b) family obligations and domestic challenges, and (c) academic preparation. The first factor found to be critical to access, retention, and success is family support and encouragement (i.e., social capital) (Falk & Aitken, 1984; L i n , 1990; Wenzlaff & Biewer, 1996). Family support and encouragement are measured i n a number o f ways, and Native American students are disadvantaged on nearly all measures. For example, students who have close family members who have succeeded in college are more likely to be successful themselves (Wenzlaff & Biewer, 1996). Few Native American college students have this advantage (i.e., cultural capital); most are the first in their families to attend college. Parents' education and household income are also strong determinants o f post-secondary access and success, and mean earnings increase with educational level (Andres, 1994; Finnie, 2001; Leek, Onge, & Lalancette, 1995). Higher parental educational attainment and higher earnings have a positive effect on children's educational attainment (Haveman & Wolfe, 1995). Such parents have, for example, more social, economic, and cultural resources to assist their children to succeed. Furthermore, parents can play an important role in motivating their children to pursue post-secondary education, and those who place a high value on higher education transmit that value to their children (Andres, 1994; Knighton, 2002). Thus, Native American students who have graduated from college tend to encourage other family members, including their children, to pursue higher education (American Indian College Fund, 2002). Family obligations and domestic challenges constitute the second factor influencing access, retention, and success for Native American students and include issues in students' lives such as childcare or transportation needs, housing problems, relationship difficulties, health problems, and employment concerns. These types o f challenges, which are widespread in Native  45  American communities, often interfere with students' attempts to pursue higher education (Boyer, 1997; Ness, 2002). Extremely high unemployment rates combined with high rates o f suicide, alcohol and drug addiction, poverty, and domestic abuse indicate conditions that often prove to be insurmountable (Boyer, 1997). When a student is faced with challenges such as these, survival, rather than educational advancement, becomes the goal (Raffo & Reeves, 2000). Academic preparation is the third major factor outlined i n the literature as critical to improving access, retention, and success i n college because mainstream elementary and high schools are typically not responsive to the needs o f Native American students. Native American students often lack fundamental academic skills, such as basic reading, writing, math, and study skills, as well as basic career information. Remediation eventually becomes a considerable barrier because students have to spend significant time and money taking developmental coursework to upgrade their skills before they can move on to college-level work (Boyer, 1997). In addition to the three main factors listed above, the literature identifies other barriers that prevent Native American students from attending or succeeding in higher education. The most prevalent o f these include distance from colleges, lack o f financial resources, lack o f motivation, low self-esteem, loneliness or alienation at school, inability to integrate in the college culture, lack o f support groups, lack o f long-range commitment or career goals, conflict o f values and customs, and poor fit (i.e., academically, socially, or culturally) between the institution and the student (Machamer, 2000; Ness, 2002; Ortiz & HeavyRunner, 2003; Shishkoff, Thomas, & A l - B a y a t i , 1999). Furthermore, once at college or university, Native American students often feel cultural discontinuity between what they learn in the classroom and what they know from their culture and tribal communities, as mainstream colleges do little to make the curriculum and teaching styles culturally relevant to students (Reyhner, Lee, & Gabbard, 1993). Again, lack o f  46  cultural capital disadvantages Native American students and decreases their likelihood o f success. The racism many Native American students experience on college and university campuses also affects their chances to succeed (Archibald & Selkirk, 1995). Instructors may have lower expectations for Native American students (Archibald & Selkirk, 1995; Shields, 1999), and the lack o f mentors or role models reduces success rates. Finally, in addition to the challenges outlined above, Native American students face many o f the same barriers that mainstream college students face, such as the complexities o f course scheduling, inadequate academic advising and counseling services, and problems accessing student financial aid (Andres & Carpenter, 1997; Archibald & Selkirk, 1995; Benjamin, 1994; Ness, 2002). M a n y o f these factors combine to make pursing a college education a low priority.  Unique Features o f Tribal Colleges that Increase Student Access, Retention, and Success Tribal colleges offer several advantages that ease the challenges facing Native American students outlined above. These advantages include convenient locations and easier access, lower cost, an appreciation o f Indigenous knowledge and culture, appropriate teaching methodologies, relevant curricula, extensive student support services, and dedicated faculty. Each o f these is elaborated upon below.  Location, Access, and Cost A primary advantage tribal colleges offer is their location. Because many Native American students can remain at home while attending a local tribal college, the financial burden of a college education is significantly reduced, and they can be close to the support o f their  47  extended families and maintain their cultural connections. This is especially important because most graduates prefer to seek employment in or near their own communities (Hayes, 1990; Ortiz & HeavyRunner, 2003). In addition, tuition fees at tribal colleges are generally lower than at community colleges, and open admission policies mean that many students who cannot meet admission requirements elsewhere, or who lack clear career ambitions, still have the opportunity to pursue further education (Hayes, 1990; Ortiz & HeavyRunner, 2003; Shishkoff et al., 1999).  Indigenous Knowledge and Culture Indigenous knowledge and culture shape tribal colleges and their philosophy o f education (Carnegie, 1997; N W I C , 2002). Although there is no universal agreement about the nature o f Indigenous knowledge, there is some consensus in Indigenous knowledge and research literature that it comprises an integrated and holistic approach to knowledge (Archibald, 1999, 2001; Battiste & Henderson, 2000; Cajete, 1994; Castellano, 2000; D e i , 2000; D e i , H a l l , & Rosenberg, 2000; Hanohano, 1999; Marker, 2000; L . Smith, 1999; White & Archibald, 1992). Battiste and Henderson (2000) quote the Royal Commission's understanding that Indigenous knowledge is "a cumulative body o f knowledge and beliefs, handed down through generations by cultural transmission, about the relationship o f living beings (including humans) with one another and their environment" (p. 42). Indigenous knowledge is "oral, experiential, holistic, and conveyed i n narrative or metaphorical language"; it "is rooted in personal experience and lays no claim to universality" (Castellano, 2000, p. 25). Castellano highlights three sources o f Indigenous knowledge: traditional teachings, which are preserved and passed down from elders; empirical observation, which is acquired through careful observation; and, revelation, which is obtained through visions  48  and dreams (p. 23). These forms o f knowledge are "qualitative and subjective rather than quantitative and objective" (p. 27) in nature and are "rooted in local cultural traditions, values, and belief systems. It is a worldview that shapes the community's relationships with its environments. It is a knowledge base that is crucial for group and community survival" (Dei, 2000, p. 79). Marker (2000) writes that "an Aboriginal approach to teaching and learning would emphasize how knowledge and sense o f selfhood come from a concrete place" (p. 41). Accordingly, values based on kinship, relationships, respect o f elders, and connections to the land are integral to tribal colleges' mission. "Students learn firmly that who they are and what they believe has great value. Rather than being a disorienting experience for students, college represents a reinforcement o f values inherent i n the tribal community" (Carnegie, 1989, p. 56). One o f the major challenges tribal colleges face is how to found the curriculum and college experience with this holistic and integrated approach to Indigenous knowledge and education.  Pedagogy One o f the most significant and, at the same time, challenging factors that contributes to tribal colleges' success is that they reflect the culture, values, and history o f their surrounding reservations, which results in greater sensitivity to the unique cultural needs o f their students (Ortiz & HeavyRunner, 2003). B y framing the curriculum i n a holistic and culturally relevant way, the instructors reaffirm the identity and values o f their students, as well as o f the surrounding Native American communities (P. Hughes, 1998; P. Wilson, 1994). The tribal college philosophy infuses familiar cultural elements and a practical approach into the post-secondary educational experience (Pavel & Colby, 1992). Battiste and Henderson (2000) note that elements o f Indigenous knowledge can best be passed on to students when  49  traditional teaching methods such as apprenticeship, ceremonies, oral stories, and practice are employed. Tribal colleges provide Native Americans with an opportunity for higher education, with smaller class sizes, in a less competitive environment (Lin, 1990; Shishkoff et al., 1999). Furthermore, whereas the focus o f the curriculum at mainstream universities tends to be on "decontextualized literal knowledge" (Kirkness & Barnhardt, 1991, p. 4), traditional, oral, or Indigenous knowledge is "interconnected" (Marker, 2004, p. 180), "is integrated in everyday life," and is "acquired through direct experience and participation in real-world activities" (Kirkness & Barnhardt, 1991, p. 4). Ideally, Indigenous knowledge forms the foundation o f the classroom and tribal college experience. Related to this natural inclination toward real-life and service-oriented experiences is a relatively new movement in mainstream colleges, and now in some tribal colleges, toward incorporating service learning into the curriculum. Service learning lends itself well to the tribal college context: Opportunities for community service work are incorporated into courses, allowing students to apply the concepts they are learning in a relevant and meaningful way (Crazy B u l l , 2004). Because tribal colleges work endlessly to provide culturally relevant experiences for their students, service learning provides the structured link between the community and the students' learning experiences. Crazy B u l l (2004) emphasizes that "these experiences can serve as the foundation for social action, which is critically needed to overcome poverty and loss in our communities" (p. 1). Likewise, in order to respond to the educational needs o f isolated students in remote communities, learning technologies, such as interactive television, have become a more accepted mode for learning (Carnegie, 1997). Independent learning and online classes are also becoming more popular, although there continue to be problems with access to computers and the Internet  50  in remote areas (Barden, 2003). Neither o f these modes for learning is ideal and, as a result, tribal colleges struggle to find ways for isolated students to access higher education using more appropriate instructional methodologies. Faculty members at tribal colleges continually experiment with culturally responsive teaching methodologies to increase their students' success.  Curriculum L i k e mainstream colleges, general education requirements at tribal colleges dictate that students take a variety o f courses to complement their core area o f study (Carnegie, 1997). O n the other hand, the incorporation o f traditional culture and knowledge throughout the curriculum, including the general education requirements, is an effort to make the curriculum more relevant, interrelated, and accessible to students (Boyer, 1990; Carnegie, 1997). The curriculum at tribal colleges, therefore, emphasizes not only academic requirements but also the cultural knowledge and traditional pedagogies o f Native American communities (Pavel & Colby, 1992). Therefore, all tribal colleges offer Native American studies courses in areas such as history, art, philosophy, botany, and local language. Restoring the use o f traditional languages is o f particular importance to Indigenous knowledge, for as M a n n (2003) emphasizes, "when a language is at risk, so is a culture" (p. xxiii). According to Amiotte and A l l e n (1989), "one o f the key reasons for the tribal college's success has been the belief and practice that students can remain Indian, can practice tribal traditions and retain tribal values and also be successful students" (p. 1).  Student Support Services Extensive student service programs are offered at many tribal colleges. These services include additional academic support, peer support, and individual and financial assistance (Stein, 2003). Students registering at a tribal college are assessed and then enrolled i n courses based on  51  their ability (Stein, 2003). M a n y students require remedial coursework, which is often self-paced and offers individual attention in a supportive learning environment (Machamer, 2000). Students are treated with respect by staff and faculty, as the well-being o f each student is o f primary importance to the tribal college and community (Kirkness & Barnhardt, 1991). Finally, tribal college staff and faculty attempt to foster i n students a strong sense o f selfesteem and resiliency, a goal-completion mentality, and an ability to "walk in two worlds" without losing personal identity (Ness, 2002). They try to prepare students academically, assist with family matters (where appropriate), and actively support students in obtaining financial aid. Because "educational attainment is an important determinant o f [future] job opportunities and relative well-being" (De Broucker & Underwood, 1998, p. 30), this support system is crucial not only to students' academic success, but to their (and their community's) well-being. Thus, tribal colleges provide a critical transition to higher education for many Native American students, whereby cultural and social capital is gained and cultural identity is affirmed (Tierney, 1999).  Faculty Faculty members play a key role in students' success at tribal colleges. Currently, most instructors at tribal colleges are non-Native because the extremely low wages, minimal benefits, and the limited number o f qualified applicants at tribal colleges make it difficult to attract Native American faculty. Tribal colleges attempt to increase the proportion o f Native faculty because these faculty provide role models for students and help them to realize the importance o f education (Carnegie, 1997). The Carnegie Foundation (1997) identified a number o f challenges instructors at tribal colleges face that limit professional development opportunities and result in high turnover rates. These include heavy teaching loads, isolation from colleagues at mainstream colleges, limited  52  funds, and under-prepared students. Generally, however, those who overcome the challenges and stay at the colleges tend to be sensitive to the culture in which they are serving and dedicated to their students. Faculty are found to be enthusiastic, supportive, caring, respectful, knowledgeable, and accessible (Carnegie, 1997). Students frequently report satisfaction with having a significant personal connection to faculty who are dedicated to their well-being (Ness, 2002). This satisfaction might be attributable to instructor-student learning relationships being more reciprocal and respectful in nature, when compared to mainstream colleges (Kirkness & Barnhardt, 1991). In addition to professional educators, tribal colleges may employ part-time instructors who are tribal members with limited formal education but who have expertise in particular areas such as language, art, history, and culture. Elders also play an important role i n the tribal colleges, either as instructors or advisors to the colleges on issues o f cultural importance (Carnegie, 1997).  Other Common Features and Challenges o f Tribal Colleges Although each tribal college has a distinctive culture and serves a unique community and economy, they all share fundamental similarities, including the challenges that they face (Stein et al., 2003). The common features o f tribal colleges include their governance structure, their focus on "strengthening economic empowerment" (Benham, 2003, p. 11), Indigenous research, and funding sources, all o f which are expanded upon below (Carnegie, 1997).  Governance A l l tribal colleges are chartered by at least one tribal government and have a board o f directors who are mostly Native American (Carnegie, 1997; Stein et al., 2003). In order to  53  become accredited, they must demonstrate administrative distance from their tribal governments (i.e., autonomy), so they can develop their own policies and avoid conflict-of-interest situations (Stein et al., 2003). Tribal colleges are endorsed as post-secondary educational institutions serving the cultural, social, and economic needs o f their communities. Their presidents and most of the administrators are well-qualified and credentialed Native American educators who are committed to the long-term success o f their institutions (Carnegie, 1997).  Economic Empowerment Tribal colleges are "seen as integral and contributing partners (in some situations, leading ones) i n the cultural and economic growth o f a community" (Benham, 2003, p. 5). Higher education is important for economic development and for improving quality o f life, the key to the social and economic growth of Native people, as well as to their continued survival. Tribal colleges empower local communities in a number o f ways (Fann, 2002). They provide higher education opportunities to tribal members that are specific to the needs o f their communities, resulting i n many graduates remaining in their communities to work. They also spend money, create employment, and provide various services to their local communities — a significant achievement, given that many reservations have unemployment rates anywhere from 4 5 % to 90% (Boyer, 1997). Even though their primary obligation is to the specific needs o f the local community, they also recognize and attempt to address the more general needs o f other Native American tribes, as well as the national and international communities (Barden, 2003). Tribal colleges are committed to providing their local communities with social and economic development initiatives through education (Barden, 2003). In the past, the primary development strategy was to attract businesses to the reservations, but most o f these initiatives were unsuccessful. Tribes learned that they needed to assess their local strengths and needs and  54  to develop culturally appropriate initiatives that could be sustained over the long term (Barden, 2003). A couple o f tribal colleges have done just that and own and operate their own businesses, which provide employment for tribal members and which they hope w i l l eventually be sold to individuals i n the community (Barden, 2003).  8  Tribal colleges provide vocational training and education so that students can return to the community as contributing and skilled members. F r o m almost the beginning, tribal colleges, like their mainstream counterparts, have provided services to the markets, including curricula that train students to fill specific market needs (Trow, 1993). For example, for the gaming industry, which has become a huge business for some tribes, tribal colleges may provide training for employees or management skills to staff. A reciprocal benefit o f this relationship is that the colleges may also receive dividends from casinos to support the college (Barden, 2003).  Research Research, or the production o f Indigenous knowledge, is an important goal o f tribal colleges. In order to conduct research, tribal colleges need financial and human resources, funding, faculty time, and advanced students — none o f which they currently have. Whereas well-endowed research universities are generally involved in the creation o f new knowledge, up until this point, tribal colleges have had to focus primarily on surviving and on transmitting and preserving Indigenous knowledge (Trow, 1993) and "rebuilding respect for traditional ways o f knowing" (Carnegie, 1997, p. x).  For example, Salish-Kootenai College has an environmental testing project.  55  These conditions are now starting to change. In 1989, with the conception o f the Tribal College Journal, a forum for sharing research and ideas among tribal colleges was created 9  (Tribal College Journal, 2005). In order to engage in research to advance Indigenous knowledge, however, in addition to adequate funding, tribal colleges can create their own research agendas and define Indigenous scholarship for themselves (Kirkness & Barnhardt, 1991). A s they progress in this direction, tribal colleges hope to be leading centers o f Indigenous research and scholarship that benefit their tribal communities socially, economically, and culturally (Stein, 2003).  Funding Undoubtedly, the biggest challenge facing tribal colleges today is the lack o f operational funding. Tribal colleges are possibly the "most underfunded institutions o f higher education in the United States" (Gipp, 2003, p. xiv). Some o f their financial challenges stem from the fact that tribal colleges are generally located i n poor communities, have low tuition fees, and do not have wealthy alumni on whom to rely (Carnegie, 1997). Furthermore, it is costly for tribal colleges to deliver education to students who are under prepared for college. Under these conditions, sources o f tribal college funding are limited to tribal funds, grants, state or local funds, private and corporate donations, philanthropic organizations (e.g., the Kellogg Foundation and Ford Foundation), and federal appropriation (Carnegie, 1997; Stein et al., 2003). Federal money provides subsistence existence to most tribal colleges through the Tribally Controlled Community College Assistance A c t o f 1978, which is administered by the Bureau o f Indian Affairs (Carnegie, 1997). Although tribal colleges rely heavily on federal  9  Initially, it was called the Tribal College Journal of American Indian Higher Education.  56  funds for core operational funding, the federal government has not, for years, provided them with the amount authorized. For example, during the 1980s, Congress appropriated $6,000 per Native American student but only provided tribal colleges with $1,900 o f that amount per student (Barbara Wright & Tierney, 1991). During the 2004-2005 academic year, Congress authorized $6,000 per Native American student and provided tribal colleges with $4,447 per Native American student (Goetz, 2005). Most commonly, no amount is provided for students who are not Native American or who do not meet the eligibility requirements o f the act; overall, 15 to 20% o f students attending tribal colleges are not Native American (Carnegie, 1997; Fann, 2002; Ortiz & HeavyRunner, 2003). A s a result o f the inadequacy o f federal support, some tribal colleges have come to rely on grant funding in order to provide for their fundamental needs. L i k e many universities that compromise their autonomy through corporate sponsorship and donations, some tribal colleges struggle to pursue their goals in the face o f requirements that are frequently determined by the grants they secure (Axelrod, 2002). Thus, the more sources o f funding a college has, the greater its institutional autonomy (Trow, 1993). Tribal colleges are i n a unique position with regard to funding. Although state funding and property taxation are the main sources o f support for community colleges, these sources are not available to tribal colleges because state governments are not mandated to provide them with funding and property taxes are not collected on trust property on reservations (Fann, 2002; Stein et al., 2003). This remains a significant problem for tribal colleges as they continue to struggle with a lack o f resources (Shanley, 2003). Tribal colleges have an additional financial challenge. They are often located i n remote areas with small population bases. Class sizes are, therefore, small, and it is more costly to  57  deliver programs to relatively few students. Moreover, because many o f their students live in even more isolated areas, the colleges must use expensive innovative solutions, such as interactive television or online classes, to reach them. Tribal colleges must also use some o f their limited resources to provide support services to these distance students (Barden, 2003). Because o f the shortage o f funds, tribal colleges typically have only "bare minimum" facilities — facilities that are frequently dilapidated, old, modular structures. Classroom space is usually limited and faculty share offices. M a n y tribal colleges also lack dining facilities, a gymnasium, student residences, childcare services, computers, or lab equipment (Carnegie, 1997). In addition to the inadequate facilities, low per-student expenditures, lack o f comprehensive student services, low staff and faculty salaries, under-funded libraries, and generally inadequate budgets continue to plague tribal colleges (Carnegie, 1997; Bobby Wright & Weasel Head, 1990). Upgrades i n the facilities and services offered to tribal college students would certainly contribute significantly to their college experience and to learning. There is a great need for tribal colleges to obtain permanent, stable funding — and the federal government has the responsibility to provide this funding for the tribes with whom it has treaties (Stein et al., 2003). A s a result, the Carnegie Foundation, in its 1997 report, recommended that the government begin to honor and provide the full amount it appropriates to each tribal college student annually. This would be a significant step forward for tribal colleges in their search for stable funding.  Conclusion Despite the lack o f basic funding, o f agreement about how to fulfill their "Indigenous" mandates, and o f student preparation, tribal colleges have had a significant impact on Native American communities. In fact, more Native American students than ever are transferring to  58  four-year institutions from tribal colleges (Ortiz & HeavyRunner, 2003). This is significant because, as Astin (1985) points out, there is sufficient evidence that completing a bachelor's degree benefits the individual significantly more than completing an associate's degree. B y assisting Native American students "who have not enjoyed full access to equal educational opportunity" (Boyer, 2003, p. 127) to transfer successfully to four-year institutions, tribal colleges are developing cultural, social, and academic capital among their students (Andres, 1994; Bourdieu, 1986; Ortiz & HeavyRunner, 2003; Tierney, 1999). In order to overcome the challenges Native Americans have faced in the last 450 years, tribal colleges continuously experiment with better ways to preserve culture and to base their curricula on Indigenous knowledge. This commitment is consistent with Castellano's (2000) hope that one day "Aboriginal knowledge w i l l resume its place as the basis o f decision making and social order" (p. 34) in Native American communities. Tribal colleges are trying to support this process and to advance the processes o f decolonization and self-determination ( A I H E C , 2004; Benham, 2003; Carney, 1999; Crazy B u l l , 2004; Marker, 2000; R. Nichols & Monette, 2003; Patterson, 1999; Shanley, 2003; Tippeconnic & M c K i n n e y , 2003). The Carnegie Foundation (1997) asserts: "Without question, the most significant development in American Indian communities since W o r l d War II was the creation o f tribally controlled colleges" (p. 1). "Tribal colleges," they state, "are crucial to the future o f Native American societies, and o f our nation" (p. 4).  59  C H A P T E R 3  ASSESSMENT IN T H EUNITED STATES — O V E R V I E W  A N D CRITIQUE  Assessment in higher education has a dual purpose: first, to improve student learning and performance (formative purpose), and second, to demonstrate to external accreditation bodies that the relationships between the institution's mission and learning outcomes (summative purpose) are evident (Trimble et al., 2000). Conventional assessment programs have tended to collect basic data such as student retention, graduation, transfer, and employment rates, success after transfer rates, and employer and graduate satisfaction, in order to guide and promote program development, long-range planning, external evaluations, and fund raising — all largely data used for summative purposes (Frye, 1999; Trimble et al., 2000); however, progressive assessment programs are emerging that are faculty- and student-driven, action-oriented, focused on mission-related performance and innovations, and attentive to student learning outcomes (Mentkowski, Rogers, Doherty, & Loacker, 2000). The first section o f this chapter presents an overview o f the nature and purpose o f assessment. It also reviews numerous criticisms o f assessment and suggests strategies for addressing them. The second section examines mainstream approaches and responses to assessment, reviews the types o f data that can be used for assessment purposes, and discusses issues specific to two-year colleges. Some o f the more innovative approaches to assessment, which foster institutional learning, are then presented. Finally, basic planning and reporting aspects o f the assessment initiative, which are essential for closing the loop and improving learning, are outlined.  60  W h a t is Assessment?  In this doctoral thesis, I use the term assessment to refer to "the systematic collection o f information about student learning, using the time, knowledge, expertise, and resources available in order to inform decisions about how to improve learning" (Walvoord, 2004, p. 2). A s an evolving concept in education, what I am calling assessment is sometimes referred to by different terms in the assessment literature. Terms such as student outcomes assessment, educational outcomes assessment, institutional assessment, and institutional effectiveness are all used in relatively interchangeable ways by different authors (Banta, 2002). I have chosen to simply use the term assessment in part for readability and in part because each o f the other terms has self-limiting connotations that do not embrace the fullest sense o f the concept as it is evolving in the literature and as I w i l l be using it. For instance, the term student outcomes assessment tends to connote individual student evaluation and obscure the importance o f assessing the educational systems that facilitate student learning, while the terms institutional assessment and institutional effectiveness tend to take the focus off student learning and are easily confused with simplistic accountability measures such as job placement rates, balanced budgets, and resource allocation. Assessment can be thought o f as action research, a scholarly endeavour used "to inform local action" (Walvoord, 2004, p. 2). The focus o f assessment today, even for accreditation bodies, is on a formative outcomes-oriented process that can be used to better understand students and for "institutional self-study, financial retrenchment, [and] program evaluation" (American Association o f Higher Education, 2004, p. 1). Angelo (1995) further elaborates on this definition: Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically.gathering, analyzing, and interpreting  61  - evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality o f higher education, (p.7)  Assessment, as it is being used here, is not about evaluating individual student performance (Seybert, 2003); rather, it is about evaluating the overall achievement o f a group o f students in order to provide feedback to students, faculty, parents, the college, policy makers, and the public about the current effectiveness and future refinement o f educational programs (Pellegrino, Chudowsky, & Glaser, 2001). The American Association o f Higher Education ( A A H E ) (2004) points out that Angelo's definition o f assessment (above) is based on the following feedback that he received from experts i n the assessment field: that the primary purpose o f assessment is to improve student learning; that effective assessment efforts are not restricted to the classroom, but include all the institutional processes that affect learning; that assessment is rooted i n a process embedded within a larger organic structure; that effective assessment programs result in a coherent, linked, and focused curriculum; and that the tension between the two purposes o f assessment — that is, improvement and accountability — must, and can, be resolved. Assessment programs identify college, program, and course outcomes that derive from the institution's mission, and they compare these intended outcomes with the actual results achieved (Seybert, 2003). Outcomes and assessment can thus be divided into three levels: college outcomes and assessment (i.e., across the entire campus environment); program outcomes and assessment (i.e., within specific programs and departments); and, course outcomes and assessment (i.e., i n specific courses and by particular instructors) ( M a k i , 2004).  62  Why do Assessment? The overarching purpose of assessment in higher education, as outlined above, is twofold: improvement and accountability (Trimble et al., 2000). A s Farmer points out, "excellence i n education does not occur accidentally" (quoted in Green & Castelli, 2002, p. 7). Assessment, at its best, is a process through which excellence can be systematically cultivated in all aspects o f higher education.  Accountability The initial impetus for assessment, however, was narrower in its focus and largely derived from the second purpose identified above, accountability. Accountability refers to summative evaluations reported to the community, funding sources, and accreditation bodies about the use o f educational resources (e.g., human, financial) in order to make decisions ( A A H E , 2004). Though assessment programs can address accountability concerns, they are broader in their scope and purpose. A s public stakeholders, legislators, accreditation bodies, funding agencies, parents, and students began scrutinizing higher education i n recent decades, they began demanding more financial efficiency and accountability, along with a more competency-based approach to education, including a change i n focus from instructor-centered teaching to student-centered learning. Pressure has also been exerted from some sectors to shift from a liberal arts focus to workforce training, resulting in less distinction between education and training. Due to concerns about the quality o f higher education, colleges have been increasingly pressed to demonstrate value-added benefits for individual students as well as for surrounding communities and potential employers (Banta, 1999; D i l l , 2003; Lieberman, 2005). Other pressures forcing the unprecedented rise in assessment and accountability include increasing competition among colleges, credential inflation, and the trend toward corporate-like  63  organizational models (Atkinson, 2004; Gaither, Nedwek, & Neal, 1995). The advent o f new technologies has impacted assessment significantly; thanks to effortless modes o f communication, it is easier now for stakeholders to voice their complaints about higher education. Expectations for information are also higher as it is simpler now to collect, analyze, and report data (Knight, 2003). N o w that higher education is accessible to the masses, and is no longer only a luxury o f the elite, there is greater demand for accountability and for identifying inefficiencies in the management o f institutions in order to reduce public spending, increase the meaningful use and management o f resources, improve planning and policy functions, and ensure academic quality ( D i l l , 2003). A s a result, in the United States, formal processes o f accreditation have become the norm and serve the following needs: Regional accreditation o f postsecondary institutions is a voluntary, non-governmental, self-regulatory process o f quality assurance and institutional improvement. It recognizes higher education institutions for performance, integrity, and quality to merit the confidence o f the educational community and the public. Accreditation or preaccreditation by a postsecondary regional accrediting agency qualifies institutions and enrolled students for access to federal funds to support teaching, research, and student financial aid. (Northwest Commission on Colleges and Universities, 2005) For two-year tribal colleges, the importance o f accreditation cannot be overstated because the acceptance o f their students' transfer credits by four-year colleges depends upon it (Knight, 2003). In the United States, six accreditation bodies are responsible for determining whether a college has a suitable mission and goals, whether it has the appropriate resources to reach those goals, and whether those resources are being used in the most effective ways. The review process involves a peer review by experts from other colleges, a self-study, and a review o f the assessment program that evaluates articulated performance outcomes ( D i l l , 2003). The entire  64  process is meant to encourage continued and ongoing self-study and improvement, to impart evaluation skills to participants, and to institutionalize assessment (King, 2004).  The Evolution o f Assessment Assessment is not a new phenomenon; faculty have been doing assessment for centuries (Walvoord & Anderson, 1998). Traditionally, assessment was confined to the classroom, and, over the last several decades, the assessment movement has evolved substantially. In the 1960s and 1970s, assessment programs began by trying to measure institutional effectiveness through readily available quantitative data such as retention and graduation rates, time to degree completion, enrollment capacities, transfer and employment rates, and performance on standardized tests. B y the 1980s, however, educators recognized that most o f these measures did not indicate whether student learning was actually occurring nor did they probe the nature and quality o f that learning. Hence, educators sought more sophisticated assessment indicators, often qualitative in nature, to assess actual student learning. This required a more integrated approach to education and assessment, with greater potential (or, some would say, risk) for educational reform (Stark & Lattuca, 1997). Moreover, it became clearer that accreditation bodies were not interested in the actual findings of assessment, but rather were interested i n determining whether or not the college had the processes i n place to collect assessment data and to use them to improve programs, services, and curricula (Walvoord, 2004). Colleges were now being called upon to collect information in a more meaningful format and to use the results to make more informed decisions (King, 2004).  65  Improvement o f Student Learning Assessment advocates assert that when assessment is done well, it emphasizes evaluation and reflection (involving students, faculty, staff, administrators, and community members) at the college, program, and course levels and results in a continuous and incremental improvement o f learning and teaching. Therefore, though external accountability pressures created the initial need for evidence that institutions o f higher education were accomplishing their goals, progressive colleges have since been working to channel these pressures into meaningful and constructive assessment programs by defining the assessment process in their own terms (Mentkowski et al., 2000). Thus, assessment programs can be used to stimulate improvements i n learning and teaching, while simultaneously demonstrating accountability commitments to the wider community (Banta, 1999). Furthermore, i n the absence o f the external accountability pressures that public institutions are increasingly facing, some private colleges have pioneered strong and innovative assessment programs precisely because they provide an effective means for pursuing educational excellence (Farmer, 1988; Mentkowski et al., 2000). A s these colleges demonstrate, when assessment is done well, it can improve student learning, clarify and strengthen the mission o f a college, improve program quality and performance, inform planning and decision making, support requests for funding, and assist i n meeting and exceeding accreditation requirements (Bresciani, 2002; Diaz-Lefebvre, 2003; Green & Castelli, 2002; National Forum on Assessment, 1995; Pellegrino et a l , 2001; Seybert, 2003; The State University o f N e w Y o r k Geneseo, 2004).  Criticisms and Concerns about Assessment  Institutional change does not come without resistance. This resistance is rooted i n a range o f concerns, including concerns about the relationship between assessment, power, and  66  administrative control; concerns that assessment programs threaten faculty independence; and concerns about the validity and purpose o f measuring higher learning outcomes. In the discussion that follows, frequently cited criticisms o f assessment are discussed and strategies for addressing them are suggested.  Power  Concerns Critical planning theorists express concerns that colleges are institutions o f power that selectively shape information, communication, participation, impressions, and attention to issues (Forester, 1989). They emphasize that information is a source o f power i n several ways: It responds to the social needs o f the institution; it provides a way for groups with fewer means to participate in the planning process; and it assists those who already have power in the system to retain it (Forester, 1989). Assessment programs require technical expertise and a central location for data and information at colleges (King, 2004). Thus, assessment programs can extend the power o f administrators by extending their possession and control o f information, including their control over who has access to what information and when (Forester, 1989). Assessment coordinators may determine what information is relevant and timely for decision-making purposes, may shape participation i n relevant decision-making processes, and may influence these processes by interpreting assessment data (Forester, 1989). In all o f these ways, assessment programs can advance the interests, agendas, and perspectives o f administrators. A t the same time, such programs can exclude the interests, agendas, and perspectives o f those who are in less privileged positions — including faculty, staff, and even students.  67  A s a result of power dynamics, some faculty are concerned that assessment protocols w i l l become rigid and w i l l be prescribed i n a top-down manner and that assessment data w i l l be used inappropriately by policy makers or administrators to rationalize administrative decisions that faculty may not support (Banta, 2002; Coffman, 2004). These concerns are well founded. H o w administrators draw attention to assessment issues and information can have significant political consequences that may include impacts on professional advancement, on workload pressures and expectations, and on resource allocations. Furthermore, administrators are themselves accountable to, or under pressure from, a range o f external entities, from accreditation bodies to legislators to powerful lobbies and special interest groups. In this context, assessment programs can become instruments o f socio-political control that powerful external agencies and groups use to reshape institutional missions and priorities and to redirect scarce resources according to their own interests and agendas.  Alternative Perspective Though many assessment programs were initiated through the demands o f external entities, such as legislators and accreditation bodies, assessment practice appears to be shifting toward an internal locus o f control with a focus on internal audiences and maximum relevancy to faculty, staff, and students (Coffman, 2004). Although assessment can serve as an instrument o f external control, the best way to ensure an internal locus o f control is for faculty and administrators to embrace assessment and take the lead in defining assessment measures and processes according to their internal values and priorities — rather than waiting for external agencies to define them according to the political pressures o f the day. Moreover, within institutions, the way administrators present information and interact with instructors can either result in indifference, suspicion, and rejection, or it can build  68  understanding, trust, and consent (King, 2004). The manner in which power is exercised is the key. Forester (1993) emphasizes that in order for planning practices to be positive and socially transformative, they must be "sensitive to the continual reproduction o f relations o f knowledge, power, trust, and attention" (p. 103). Individuals in positions o f power can strive to present information in an open, unbiased, and user-friendly manner; commit to using the results o f the assessment program; create opportunities for interdependence to enhance collaboration and to build trust; anticipate conflicts, obstacles, and concerns; develop keen listening and speaking abilities; cultivate a network o f meaningful relationships; and respond in sincere, respectful, cooperative, and nurturing ways (Forester, 1989; K i n g , 2004). A collaborative response enhances a sense o f collective identity and purpose, an approach that results i n better solutions and has the potential to transform working relationships (Forester, 1989). Meaningful participation in the process, whereby faculty's perspectives are validated, increases commitment to the process and, thus, chances for the college's success. In order to do this, it is essential that adniinistrators relinquish some o f their power and collaborate more with faculty (Coffman, 2004). Issues o f power w i l l always remain, but, i f administrators sincerely listen to faculty and staff, this power can be used to transform educational institutions into models o f decision making built on shared values and interests (Forester, 1989; K i n g , 2004). Thus, the individuals involved have ethical responsibilities to not silence voices and to understand how their presentation o f analyses influences power relations within the organization, so that they can empower the faculty, students, administrators, and staff in positive ways (Forester, 1989).  69  Academic Freedom and Evaluation  Concerns Some faculty have raised concerns that assessment findings may be used to evaluate and reprimand them or their students. A related concern is that assessment processes w i l l increase external control over curriculum and testing. Within the new assessment regime, critics assert, some faculty w i l l try to anticipate what external bodies are looking for and adapt their teaching to what they believe w i l l be measured. This kind o f behaviour may discourage instructional creativity and risk-taking as well as increase competition among programs (Ewell, 2002; Fisher, Rubenson, & Rockwell, 2000; Lieberman, 2005; Rodrigues, 2002).  Alternative Perspective Assessment proponents point out that assessment processes do not give administrators the authority to advise faculty on how to teach (Lieberman, 2005). Instead, the goal o f assessing student learning is to assist faculty, as a group, to improve the learning process; the data gathered under these conditions can be used to improve curricula, programs, and instruction from the bottom up rather than the top down (Walvoord, 2004). In an effective assessment program, the emphasis is on the entire group o f instructors and when evidence o f poor student learning in a particular class emerges, it can and should be addressed with a supportive and encouraging response from the department or college. Moreover, in a two-year college context where teaching is the primary responsibility o f instructors, proponents believe that assessment makes instructors more accountable and that some level o f accountability is healthy, because it can result in more meaningful learning experiences for students (Walvoord, 2004). Because assessment programs use aggregate rather than individual data about learning, and do not  70  associate students' names with their scores, fears about student privacy and evaluation are unfounded (Lieberman, 2005). Finally, because accreditation bodies' focus is now on individual college-generated outcomes (ideally determined by the faculty), the risks o f discouraging instructional creativity and "teaching to the test" are diminished.  Resources  Concerns M a n y faculty also assert that the time, money, and effort required to learn about and implement complicated assessment processes could more profitably be directed toward supporting conventional teaching, learning, and service activities (Fisher, Rubenson, & Rockwell, 2000; Rodrigues, 2002; Walvoord, 2004; Zorzi, M c G u i r e , & Perrin, 2002). In addition, they argue that faculty already grade students and this should be an adequate form o f evaluation (Rodrigues, 2002). Finally, because faculty often receive no compensation or 10  reward for learning about and implementing assessment processes, many are not motivated to devote the extra time necessary to carry out these added responsibilities.  Alternative Perspective These concerns w i l l be addressed from two perspectives: those o f administrators and faculty, both o f whom are essential to the implementation o f a successful assessment program. In both cases, a transformational change is needed: administrators "toward a culture that more  Refer to the section titled Grades in chapter 3 for a discussion about grading and its relationship to assessment.  71  freely embodies the principles o f a learning organization" (Banta, 2002, p. 284) and faculty toward a culture o f active responsibility for documenting and improving student learning. Assessment scholars agree that administrators must provide visible advocacy as well as real material support for assessment i f it is to succeed. The value they place on it can be shown by their taking an active role and interest in assessment; collaborating with faculty, staff, students, and others involved in the process; allocating financial, technical, and human resources to assessment; and referring regularly to the assessment process and its results in reports and presentations to both internal and external audiences (Center for the Study o f Higher Education, 2002). In an effective assessment climate, administrators only hold individuals and programs accountable for what they have control over (i.e., monitoring improvement rather than comparing performance), and they use the results to improve the collective benefit o f the programs, rather than to encourage competition between programs (Ewell, 2002). They recognize and value feedback, ideas, and input from staff and faculty, and they incorporate these suggestions into their plans and decisions. Furthermore, time-consuming assessment processes and meetings are streamlined and simplified, and administrative obstacles are reduced so that instructors' time may be used wisely (Rodrigues, 2002). For example, assessment processes should only be designed to provide information the college can use; not to collect data for which there is no identified use (Banta, 1999; Gray & Goodman, 2003; J. Nichols, 2002; Seybert, 2003; Walvoord, 2004). To facilitate the assessment process, administrators may also need an assessment coordinator who has the responsibility to support and coordinate institution-wide assessment activities; provide expertise and consulting for all educational programs and administrative  72  departments; collect data in an efficient manner; and provide a centralized location for the storage, analysis, and dissemination o f assessment results. In addition, administrators also need a process for evaluating the effectiveness, including the cost-effectiveness, o f the assessment program itself (J. Nichols, 2002). A s an increasing number o f colleges attempt to implement assessment programs ( E l Khawas, 1995), faculty often emerge as the main source o f opposition for the reasons discussed above as well as others (Cohen, 1994). For instance, traditionally, faculty have focused on the process, not the product, o f teaching. Assessment is a cooperative process that requires agreement about mission, program goals, and learning outcomes; methods and standards o f assessment; interpretation o f results; and use o f information to improve the learning experiences for students. M a n y faculty, however, are used to teaching in relative isolation and are not wellprepared for this type o f collaborative approach (Banta, 2002). In addition, developing assessment programs takes time and energy, and many faculty already feel overburdened. A s a result, faculty often prefer to let administrators deal with assessment (Walvoord, 2004). Yet, proponents o f assessment agree that a successful assessment program needs to be driven by strong faculty leadership, implying primarily that faculty participate fully in the design o f learning outcomes (Seybert, 2003). To be effective, faculty can take ownership o f assessment and embrace it as an intrinsically valuable developmental process whereby teaching and learning can be continually improved through evaluation, reflection, and identification o f needs for change. A sense o f ownership, and hence cooperation, is enhanced when faculty and their colleagues themselves create meaningful assessment processes, rather than having them imposed by the administration (Rodrigues, 2002). This requires faculty being involved daily in  73  assessment processes that reflect what they value, such as their teaching, disciplines, and research (Rodrigues, 2002). Faculty ownership and participation result in a continuous process o f improved learning and teaching. Toward this end, strategies to make the assessment process more streamlined include integrating assessment into the curriculum; embedding assessment i n classroom assignments to serve both grading requirements and collective performance purposes through the use o f rubrics (Ewell, 2002); determining in advance clear learning outcomes, high standards, and clear criteria for levels o f achievement; agreeing upon the tools that w i l l be used consistently to measure success; pre-determining a process o f using this information i n meaningful ways to improve teaching and learning; communicating assessment results frequently; connecting "assessment, as a form o f systematic inquiry, with the scholarship o f teaching" (Banta, 2002, p. 288-289); and using the results to improve programs and student learning experiences. Finally, assessment scholars agree that it is wise to start the process with interested and supportive faculty; to provide them with the necessary opportunities, incentives, and material resources to learn about assessment; and to celebrate, credit, thank, and reward their efforts and achievements publicly through institutional processes and practices (e.g., with stipends, promotion, tenure, release time, etc.) (Banta, 1999, 2002; K i n g , 2004; J. Nichols, 2002).  Measuring Higher Learning  Concerns Faculty raise many legitimate concerns about the validity o f measuring educational outcomes, as well as the socio-political consequences o f the movement toward greater accountability. Most notably, many instructors think it is impossible to measure the important  74  goals o f higher education and lament the skewed picture that results from too great a reliance on quantitative data (Rodrigues, 2002). They have a point. Too often, during the planning processes in which assessment approaches have been developed, meaningful outcomes and activities have been excluded in favour o f more easily quantifiable measures. Not only is reliance on quantitative data easier, it is politically popular in a society that is preoccupied with measurement and testing (Banta, 2002; Fisher, Rubenson, & Rockwell, 2000). A s a result, many faculty argue, quantitative indicators have a disproportional influence relative to qualitative indicators that may be more difficult to measure, but more meaningful or important ( A . Wilson & Cervero, 1997). This "technical rational" approach, based on scientific problem-solving, limits understanding o f what is considered to be appropriate theory or practice and excludes certain approaches to education and problem-solving that are not quantitative ( A . W i l s o n & Cervero, 1997). A s Axelrod (2002) states, "traditional teaching methods based mainly on memorization, frequent testing, and other quantitative assessments may satisfy the current craving for educational 'accountability,' but on their own they insufficiently cultivate the life o f the m i n d " (p. 124).  Alternative Perspective Though faculty voice concerns about measuring higher types o f learning in a meaningful way, proponents o f assessment point out that instructors are frequently called upon to use their professional judgment to evaluate the work o f their colleagues, students, and themselves by criteria accepted in their fields. Therefore, it is not unreasonable to expect faculty to use the same kind o f expert judgment to assess how their students' work compares to the higher expectations o f the college (Ewell, 2002; Walvoord, 2004). Furthermore, faculty can transform the discourse about accountability by devising meaningful ways to demonstrate quality or  75  student performance and creating "processes that develop a sense o f common purpose and shared accountability" (Gaither et al., 1995, p. 4). Outcomes discussions among faculty often initiate insightful and meaningful conversations and are " i n service o f understanding the discipline and how it might and should affect students. Ending with a potentially measurable outcome may be the least important result" (University o f Washington, 2005, p. 1). In addition, because education is a fundamental contributor to collective social progress, the responsibility o f faculty, and everyone else engaged in the educational enterprise, is to devote their talents and insights to the admittedly difficult, but nonetheless necessary, task o f continually improving its quality. A s E w e l l (2002) points out, "large-scale assessment o f higherorder collegiate abilities can be an important tool o f social policy, i f it is directed toward determining gaps in the nation's store o f 'educational capital' and filling them" and not used for "rating and governing institutional performance" (p. 14). Therefore, the discourse about the value o f educational assessment "must recognize the role o f liberal education i n furthering the public good" (Fisher, Rubenson, & Rockwell, 2000, p. 24) and the importance o f education in instilling social values and responsibility. Through their contributions, faculty can contribute to more creative and flexible assessment programs, incorporating qualitative as well as quantitative measures, which may result in what W i l s o n and Cervero (1997) refer to as "emancipatory learning" (p. 100) and in the emergence o f alternative planning models based on the ideals o f social justice and social transformation.  76  Other Measurement Issues  Concerns Other related concerns faculty often raise about measurement are the following: First, assessment concentrates on tools o f the trade, such as the current focus on rubrics, rather than on important questions (Ewell, 2002). Second, colleges frequently focus on target outcomes rather than the processes (Harris, 1998). Third, it is difficult to develop indicators and methods that capture meaningful data or to agree about definitions o f outcomes and connect measurable outcomes to a single educational process (Cohen, 1994; J. Nichols, 2002). Fourth, quantitative measures can reduce the complexity o f situations, are often inaccurate and inconsistent when compared among colleges, are burdensome, and are unable to provide meaningful information that allows colleges to draw useful conclusions (Fisher, Rubenson, & Rockwell, 2000). Put another way, Coffman (2004) characterizes quantitative measures as those that focus on the "logical versus the creative, the planned versus the emergent, the revolutionary versus the transformational, the strategic fit versus reaching with a stretch, and the pursuit o f a strategy versus going after effectiveness" (p. 21).  Alternative Perspective Though concerns about over-reliance on quantitative data are valid, approaches to assessment are evolving in a direction that makes these concerns less relevant. The sociopolitical discourse mentioned above has had a profound impact on the evolution o f assessment. Banta (2000) summarizes how the current thinking developed. These concerns, she says, forced practitioners to sharpen the philosophical grounding o f the movement — rooting it in the tenets o f scholarship and the process o f teaching and learning. It also reemphasized that the evidence used by assessment must always rest upon a peer-based  77  conrmunity o f judgment (American Association for Higher Education, 1992; Mentkowski, Astin, E w e l l , & Moran, 1991). Finally, the debate forced explicit recognition o f the fact that evidence is consistently constrained by the context i n which it is generated (Mentkowski & Rogers, 1988) and by the uses to which it is put (Messick, 1988). Epistemological issues o f this kind thus remain at the heart o f the movement and remain healthily and vigorously contested (Ewell, 1989; Harris & Sansom, 2001). But protests based solely on principle or politics have steadily diminished, (p. 18) Thus, just as the focus o f the debate has shifted, so too has the focus o f assessment. Today, assessment tends to focus on the institution's goals and a determination o f whether students are attaining those goals — rather than on how outcomes compare to other colleges. This position has resulted in assessment becoming a more meaningful endeavour. Fortunately, accreditation bodies are supporting this approach, replacing a focus on rigid data-driven reports with a focus on encouraging and expecting colleges to use their data, especially qualitative information, to improve curricula and programs (Banta, 2002). Locally developed, authentic assessment approaches are now preferred (Fisher, Rubenson, & Rockwell, 2000; Stufflebeam, 2001). Alverno College, discussed later i n this chapter, provides a leading example o f the effort to apply authentic qualitative indicators to the learning process.  Student Learning  Concerns Another concern some faculty have cited is the lack o f evidence that assessment improves learning (Ewell, 2002). A related concern is that the policies resulting from assessment w i l l not improve institutional climate.  78  Alternative Perspective It is true that assessment processes, as relatively recent developments in higher education, have not yet yielded much data to support the claim that they improve learning; however, assessment advocates would argue that the only way to determine whether such evidence w i l l be forthcoming is to experiment with assessment approaches and monitor the results. When properly conducted, assessment may well be a valuable tool for improving program quality and performance and, thus, enhancing students' learning. Furthermore, assessment scholars emphasize that, i f students are provided with numerous, varied, and meaningful opportunities to practice skills and receive feedback, the assessment process itself can improve learning (Mentkowski et al., 2000; Stufflebeam, 2001). A t Alverno College, for example, faculty assert that when assessment is done well, it results i n learning that lasts (Mentkowski et al., 2000). For a more thorough discussion o f this topic, refer to the Assessment as Learning section in chapter 4.  Dual Purposes o f Assessment  Concern Another source o f tension over assessment stems from its dual purposes:  improvement,  which is internally designed in order to improve an institution's own teaching and learning, and accountability,  which is externally imposed (through state mandates, government, and  accreditation bodies ) to judge how well an institution is meeting the external goals o f these 11  agencies. These two purposes can be contradictory and incompatible (Ewell, 2002).  11  Faculty sometimes see university administrators, such as presidents and deans, as external.  79  Alternative Perspective Assessment scholars assert that this issue is resolvable. To begin with, it is necessary and useful to recognize the nature o f the different purposes o f formative (improvement) and summative (accountability) evaluation. Both types o f evaluation are essential, but they must be dealt with as separate and distinct processes. Second, formative information can be extracted from the existing processes designed to acquire summative information. For example, a grade provides a summative evaluation o f a student's work; on the other hand, an assessment rubric provides formative information that can lead to an immediate focused response and remediation to improve learning (Ewell, 2002). B y recognizing the tension between assessment for improvement and assessment for accountability, by dealing with them i n separate processes, and by extracting formative information from summative processes, assessment processes may be streamlined and tensions between them resolved. Accreditation bodies are beginning to assist in the process o f resolving this tension. In the last couple o f years, they have shifted their focus from requiring prescribed data-driven reports (i.e., addressing external concerns) to focusing on the fulfillment o f each college's mission and goals and how the college uses its assessment results to improve student learning (i.e., addressing internal concerns).  Assessment is a Fad  Concern Some faculty are still convinced that assessment is just a fad and w i l l disappear (Ewell, 2002).  80  Alternative Perspective Proponents o f assessment insist that this is not the case and that the assessment movement w i l l continue to gain momentum (Ewell, 2002). This momentum can be detected i n the assessment literature where the debate is no longer about the pros and cons o f assessment but about better means o f accomplishing its purposes (Banta, 2002). Banta affirms, "just as assessment revitalized accreditation, accreditation's insistence on assessment has kept the assessment movement alive and thriving" (p. 252). Politicians, external stakeholders, the media, and the market are all obsessed with data and information (Walvoord & Anderson, 1998). Furthermore, Banta (2002) points out, "assessment has been a point o f confluence for a raft o f issues in education" (p. 252) (e.g., the use o f technology as an instructional mode, outcomesbased curricula, writing-across-the-curriculum, learning communities, service learning, and problem-based learning), all o f which must demonstrate their effectiveness. A l l o f these developments have served to shift educational focus from teaching to learning and to propel the assessment movement forward (Ewell, 2002).  Benefits of Effective Assessment Processes  Despite the concerns and criticisms discussed above advocates believe that good assessment practice can be used to improve colleges and to meet the needs o f accreditation bodies and other stakeholders. Assessment is an ever-evolving, dynamic process, that can be adapted to the needs o f the college culture (Banta, 2002; M a k i , 2004; J. Nichols, 2002; University o f Wisconsin, 2004). When done well, advocates insist, the benefits o f assessment are numerous: It initiates meaningful conversations at all levels within a college; provides an opportunity for creating a shared vision for the future o f the college, based on common values; redirects resources towards priorities outlined in the mission and goals; increases the college's  81  responsiveness to the needs o f the coirirnunity; builds cohesion, collaboration, relationships, and trust among faculty, staff, and administrators; re-values teaching, service, and students; improves the instructional capacity o f the college as well as its public image; increases students' confidence in the college and in themselves; provides the basis for college planning and budgeting decisions; provides financial or reputational rewards at the individual, department, or college-wide level; and demonstrates accountability, the responsible use o f limited resources, to the public (Fisher, Rubenson, & Rockwell, 2000; Fisher, Rubenson, Rockwell, Grosjean, & Atkinson-Grosjean, 2000; Lieberman, 2005; Walvoord, 2004; Zorzi et al., 2002). Assessment data can be used to improve everything from curriculum and course content, pedagogy, assessment tools, internships, and faculty-student interactions, to facilities, course staffing and scheduling, class size, inclusion o f students i n faculty research, and student advising (Walvoord, 2004, p. 62). Indications o f the assessment movement's advancement include the creation o f teaching and learning centers and greater faculty interest i n teaching and learning scholarship (e.g., the promotion o f assessment in various disciplines' associations, publications of journal articles, and presentations at conferences) (Ewell, 2002). Based on an explicit understanding and analysis o f the critiques, Banta's (2002) view reaffirms the significance o f the assessment movement, which, she says, has "begun to build a whole new infrastructure for teaching and learning, for improvement and accountability.. .Building that infrastructure takes time. It progresses at a glacial pace. But it reshapes the whole landscape" (p. 257).  M a i n s t r e a m A p p r o a c h e s to A s s e s s m e n t  M a n y assessment advocates refer to the following set o f nine principles that they believe form the foundation o f good practice for assessing student learning:  82  1. The assessment o f student learning begins with educational values. 2. Assessment is most effective when it reflects an understanding o f learning as multidimensional, integrated, and revealed in performance over time. 3. Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes. 4. Assessment requires attention to outcomes and also to the experiences that lead to those outcomes. 5. Assessment works best when it is ongoing, not episodic. 6. Assessment fosters wider improvement when representatives from across the educational community are involved. 7. Assessment makes a difference when it begins with issues o f use and illuminates questions that people really care about. 8. Assessment is most likely to lead to improvement when it is part o f a larger set o f conditions that promote change. 9. Through assessment, educators meet responsibilities to students and to the public. ( A A H E , 1992, p. 1)  In varying degrees and in varying ways, colleges are attempting to integrate these principles into their assessment o f student learning. In addition to agreement about integrating these principles, a consensus about the steps involved in the development o f an effective program o f assessment appears to be emerging in the assessment literature (Banta, 1999; J. Nichols, 2002; Seybert, 2003; The State University o f N e w Y o r k Geneseo, 2004; Walvoord, 2004). The first step is to either establish or update the college's mission statement and articulate college and program outcomes. Outcome statements describe "what students should be able to demonstrate, represent, or produce based on their learning histories" ( M a k i , 2004, p. 60). Next, indicators and standards are established to operationalize these outcomes, methodologies are selected for the data collection, evidence is gathered o f how well students are attaining the outcomes, and the information is analyzed.  83  Finally, administrative and communication systems are designed to ensure the findings are actually incorporated into decision-making processes and used to refine policy and practice at the college, program, and course levels.  Types of Information Used for Assessment Purposes  Effective assessment programs use three types o f data —- often referred to as direct indicators, indirect indicators, and institutional data — for assessing success at the college, program, and course levels (Palomba & Banta, 1999). Whereas direct indicators require that students demonstrate their learning through, for example, essays, capstone projects, tests, and presentations, indirect indicators ask students to reflect on their learning through, for example, graduate or student satisfaction surveys, interviews, and focus groups. Institutional data, on the other hand, are institution-level measures that do not necessarily indicate student learning but do reflect the overall condition and effectiveness o f the college and may include, for example, retention and graduation rates, student-faculty ratios, and enrollment trends (Ewell, 1997; Walvoord, 2004). It is worth noting that in the assessment literature, there is some disagreement and inconsistency regarding which specific indicators fall within each o f these three broad categories (and even some disagreement regarding the utility o f the categorization scheme itself).  Direct Indicators It is widely accepted that certain teaching strategies — such as self-reflecting, practicing and repeating, applying concepts to a relevant context, teaching material to peers, writing about a subject, and asking essential questions — foster deep learning ( M a k i , 2005). It is also known that "what and how students learn depends to a major extent on how they think they w i l l be assessed" (Biggs, 1999, p. 141).  84  Traditional and Authentic Assessment Tools Direct methods o f assessing student learning tend to fall into two general categories: traditional tests (e.g., exams) and authentic assessment tools. Both o f these approaches attempt to assess the observable performance o f students. Table 3.1 provides examples o f both traditional tests and authentic assessment tools that can be used directly to assess student learning, at the college, program, or course levels (Cai State San Bernadino, 2004; Gipps & Stobart, 2003; Green & Castelli, 2002; Knight, 2003; M a k i , 2002; J. Nichols, 2002; Palomba & Banta, 1999; Seybert, 2003; The State University o f N e w Y o r k Geneseo, 2004).  85  Table 3.1 Examples o f Direct Methods for Assessing Student Learning Traditional tests  Authentic assessment tools  Certification exams  Apprenticeships  Cognitive assessment tests  Capstone projects or experiences  College competency tests  Case studies  Comprehensive tests / exams  Competency-based curricula  Critical thinking tests  Essays (written projects, pre and post)  Entrance tests  Internships (externally reviewed)  General knowledge tests  Interviews  Graduate entrance exams (e.g., G R E )  Job performance  Licensure exams  Journals  National exams  Juried reviews  Oral communication tests  Observations o f student behavior or skills  Placement tests for entering students (e.g., math, writing or reading)  Oral presentations and exams  Pre- and post-tests for attitudes and mastery o f knowledge Professional exams Quantitative problem solving tests  Peer evaluation Performance-based mastery tests Portfolio system Presentations  Standardized tests  Projects (e.g., abstracts, advertisements, brochures, budget with rationale, research)  Writing proficiency and reading competency tests  Reflection logs Self-assessment Simulation exercises Thesis / major project / doctoral thesis Video and audio tape evaluations (pre/post) Workplace competency  Traditional tests may be standardized or locally designed and may be o f different types (e.g., multiple choice, fill-in-the-blank, short answer, true or false, etc.). Although they may or may not reflect a student's ability to truly understand or apply the concepts and material, some  86  scholars believe that traditional tests provide information about cognitive outcomes (Palomba & Banta, 1999). Authentic assessment tools, on the other hand, may provide students with the opportunity to demonstrate behaviours and performance in more "real-life" and applied situations. These authentic assessment tools ideally measure the knowledge and abilities expected by professionals in the students' respective fields (Wiggins, 1998). The main disadvantages with using authentic assessment tools is that they require a large investment in resources and in time setting up the systems; they take more time and effort to score; they may not be reliable; they often lack comparable norms; and they usually cover a narrow range o f skills (Stufflebeam, 2001).  Grades The relationship between grades and assessment is not as straightforward as it might first appear. Typically, completion o f a course, or receipt o f a grade, does not in itself provide evidence o f learning or meeting a learning outcome (Strain, 2003). The fact that a student obtains an A in a course does not mean that he or she has gained understanding or knowledge as a result o f the course. For instance, the student may have entered the course with the understanding or knowledge required to earn an ,4. Grades do not necessarily measure how much students know, value, or are able to do; how much they have learned due to a course; or, i f learning has occurred, what elements impacted their learning most (Center for the Study o f Higher Education, 2002). Furthermore, because the way grades are determined can vary greatly among instructors and colleges, they cannot be used for comparison purposes. Nonetheless, in a tribal college setting, there are two circumstances when it is generally acceptable to use the actual course grades for assessment purposes. The first is when students  87  move from developmental to college-level classes. The second is when students transfer from a two-year tribal college to a four-year university. In both these instances, grades may be used to assess successful transfer from one level to the next (J. Nichols, 2002; Walvoord, 2004). B y comparing a student's performance at the next level with others at that level, the tribal college can get a sense o f how well prepared their students are and how effective their grading system is. A n effective grading process can serve several purposes. It can enhance evaluation o f the student, can extend communication between the instructor and student, can motivate students to learn, can organize course concepts, and can be used to improve teaching (Walvoord & Anderson, 1998). Grading, however, is a complex process, is never totally objective, and can greatly affect student learning, both positively and negatively. It is a "socially constructed, context-dependent process" which, at its best, "can be a powerful tool for learning" (Walvoord & Anderson, 1998, p. 10). If the grading process is approached using rubrics (e.g., see Table 3.2), where criteria are made explicit, are evaluated using a scale, and are used to foster improvement, then "the grading process is an excellent basis for direct assessment o f learning" (Walvoord, 2004, p. 15). Rubrics articulate criteria that instructors use to evaluate a specific assignment, and these results can be used to communicate and compare aggregate results (Walvoord, 2004). A s a result, instructors can determine specifically where students are having the most difficulty and whether, after adjusting teaching strategies, students improve on a specific criterion over quarters. Moreover, in terms o f assessment, rubrics make grading public and analyzable (Walvoord & Anderson, 1998).  88  Table 3.2 A Section o f the N W I C Rubric for Evaluating Writing Skills (Draft) Level of Proficiency Criterion  Beginning (1)  Developing (2)  Outcome a: The student writes standard English 1. Idea and Writes with Partially focuses content unclear purpose or on topic with central theme. minimal or no Does not clearly support of define or support position. Writing position on topic. is basic, too Uses limited or general for the disconnected reader to develop details that disrupt a clear the unity of the understanding. paper.  |  Accomplished (3)  Exemplary (4)  Maintains clear focus throughout the paper with sufficient . appropriate details indicating awareness, knowledge, and insight.  Writes clearly and with focus; relevant details support the central theme.  2. Organization / structure  Writes with organization that is unclear or inappropriate to the thesis; lacks transitions between ideas.  Writes with some signs of logical organization; may include abrupt or illogical shifts and ineffective flow of ideas. Makes few transitions between ideas.  Supports thesis and purpose through organization and paragraphing; most transitions are appropriate, but sequence of ideas may need improvement. Reiterates introductory elements in conclusion.  Provides clear introduction and reinforcing conclusion. Orders writing logically with effective transitions, providing sufficient information in the appropriate places.  3. Word choice  Chooses nonspecific or distracting words that limit meaning. M a y include slang and colloquialisms. Writes with a minimal grasp of standard writing conventions; numerous errors impair readability.  Chooses ordinary words using adequate verbs, nouns, adjectives, and phrases.  Chooses correct words that result in clarity.  Chooses interesting, specific and accurate words that contribute to communicating the writer's purpose.  Writes with a basic grasp of the standard writing conventions; occasional errors may impair readability.  Writes with a good grasp o f standard writing conventions: capitalization is proper; punctuation is smooth and enhances meaning; spelling and grammar are essentially correct.  Writes with a strong grasp of the standard writing conventions; all conventions are properly applied.  Produces writing that looks untidy and does not follow basic formatting rules (e.g. margins, headers and subheaders).  Produces writing that looks fairly neat but violates some formatting rules.  Produces writing that looks neat but violates one or two formatting rules.  Produces clean, neat, and easily read document in which the form and presentation of the text enhance the written message.  4. Writing conventions: Grammar / spelling / punctuation  5. Presentation / formatting  89  In order to use the grading process for the direct assessment o f learning, Walvoord (2004) advises instructors to ensure that the assessment tool actually measures the learning outcomes. It must state explicitly in writing the criteria for evaluating student work in sufficient detail to identify students' strengths and weaknesses; develop systematic ways o f feeding information about student strengths and weaknesses back to decision makers.. .and using that information for programmatic improvement, (p. 15) Using the grading process for the direct assessment o f learning can be accomplished in a number o f ways. For example, in a supportive faculty meeting environment, instructors can present their students' assignments, the written criteria (e.g., rubric) (see Table 3.2), and the class's aggregate scores compared to previous quarters (see Table 3.3) for feedback. Faculty can then make recommendations for improving student learning, and someone can take minutes o f the meeting to record the exchange. The instructional changes that result from these sessions can later be documented.  Table 3.3 M e a n Aggregate Scores from a Native American History Class Assignment Assessing the N W I C Outcome "Writing Standard English" Quarter Fall 2004  Winter 2005  Spring 2005  Fall 2005  Winter 2006  1. Ideas and content  4.0  3.5  3.7  3.5  4.0  2. Organization / structure  2.0  2.5  3.0  3.2  3.5  3. Word choice  3.0  3.5  3.5  3.7  3.9  4. Writing conventions: Grammar, spelling, punctuation  3.7  4.0  4.0  4.2  4.4  5. Presentation / formatting  3.0  3.0  3.2  3.4  3.5  Criterion  Note. Adapted from Walvoord (2004).  90  Table 3.3 illustrates how the grading process may be used for assessment purposes. In this example, instructors assessed the N W I C outcome writing standard English in this course over a number o f quarters. The second criterion, organization and structure, had the lowest score in the fall quarter o f 2004. The instructors focused more o f their attention on this criterion during the next few quarters, and results indicate modest improvement each quarter. O f course, grades also serve a purpose i n evaluating individual student performance in a course because they summarize the level o f the student's achievement o f a set o f outcomes. Grades inform students about how well they perform in a class compared to other students, yet without clear and detailed rubrics they do not indicate in which areas the student is strong or weak (Palomba & Banta, 1999). F o r instance, i n Table 3.4, although all four students varied significantly on their final grades, all were successful i n demonstrating criterion 3 (word choice).  Table 3.4 Relationship between Individual Student Grading and Assessment Student A  Student B  Student C  Student D  Average o f criterion  1. Ideas and content  5  3  4  3  3.75  2. Organization / structure  5  2  2  3  3  3. W o r d choice  5  5  5  5  5  4. Writing conventions: Grammar, spelling, punctuation  5  3  4  4  4  5. Presentation / formatting  4  2  3  3  3  Total  24  15  18  18  Grade  A  C  B  B  Criterion  Note. Adapted from J. Nichols and K . Nichols (2000, p. 43).  91  Therefore, i f we want to understand student achievement o f a certain learning outcome, we can gather information specific to that learning outcome, repeatedly, over time (J. Nichols & Nichols, 2000). Some scholars think grades may be inappropriate i n Native American institutions, as grades tend to single out individuals from the group (Fixico, 2000). W i t h the strong emphasis i n Native American culture on cooperation, rather than competition, and the unity o f the community and group, it may be important for tribal colleges to develop "learning-oriented" rather than "grade-oriented" students (Walvoord & Anderson, 1998). Nonetheless, because tribal colleges must respond to the demands o f the universities to which their students transfer, it is important for tribal colleges to determine ways to use the grading system to their advantage.  Classroom Assessment Techniques Some direct methods for assessing student learning are actually not graded. For instance, short, frequent, ungraded attempts to assess student learning, referred to as classroom assessment techniques provide immediate in-class feedback from students about which concepts are clear and which are not (Angelo & Cross, 1993). One example o f a classroom assessment technique is called the "muddiest point," where the instructor asks the students at the end o f a class to write down, in one minute, the concept that was least clear to them during the class. The instructor gathers the sheets, reviews the responses, and then responds to them the following class (Angelo & Cross, 1993). These formative assessment techniques provide instructors with the opportunity to make adjustments to courses mid-stream (or even mid-class) and to build trust and relationships with their students (Lieberman, 2005). The use o f other formative assessment techniques, such as self-assessment and peer reviews, are also useful and tend to be intrinsically rewarding for  92  students. Although it is difficult to tie these efforts formally to program assessment, assessment scholars are convinced that employing these strategies improves student retention and engagement and provides instructors with feedback as to what students are or are not learning (Banta, 2004; Walvoord & Anderson, 1998).  Curriculum M a p A n effective tool for determining where each o f the college outcomes is being assessed is the curriculum map. B y listing all program or college course requirements, this simple matrix summarizes the level to which students are expected to master each o f the outcomes in each o f the required courses (see Table 3.5).  Table 3.5 Example o f a Section o f a Curriculum M a p College outcome 1. Cultural Required course E N G L 101  a B  M A T H 102 B I O L 101  b  2. Written comm. a  b  B  B  B  3. Oral comm.  4. Technology skills  a  a  b  B  H I S T 201  I  d  e  a  b  B  6. Reading skills a  b  B B  B B  B I  f B  B B  P O L S 101  c  B B  B  b  5. Quant. skills  I  B I  Note. Adapted from Walvoord (2004). The level to which each goal is assessed in each class is identified: B = beginner, I = intermediate, A = advanced.  93  The curriculum map provides an efficient and useful way to identify gaps i n the program where outcomes may be neglected. For example, the abbreviated curriculum map in Table 3.5 indicates that Reading skill 6b — extending students' vocabulary through reading — is not being assessed in this curriculum. A draft list o f N W I C outcomes is posted in chapter 6 (College Outcomes).  Indirect Indicators The second category o f data used for assessing student success is indirect indicators. Indirect indicators provide information about students' perceptions o f their learning experiences. Examples o f indirect indicators o f student learning are listed in Table 3.6, according to the enrollment status o f the student (Alfred, E w e l l , Hudgins, & McClenney, 2000; Banta, 1999; C a l State San Bernadino, 2004; C S H E , 2002; Green & Castelli, 2002; Lancaster et a l , 2003; M a k i , 2002; Seybert, 2003). Although indirect indicators can provide valuable information about the strengths and weaknesses o f a college, program, or course, Palomba and Banta (1999) argue that these data must be combined with activities that directly assess learning in order to produce reliable assessment results. Thus, indirect measures can support results o f direct measures or supplement them when they are difficult to measure.  94  Table 3.6 Examples o f Indirect Methods for Assessing Student Learning  Current students:  Graduates:  •  Campus climate surveys  •  •  Course evaluations by students  •  Engagement surveys  •  Evaluations o f programs and services by students  •  Exit interviews and focus groups  Graduation follow-up studies and surveys: reporting satisfaction with college program and career success / advancement and performance at subsequent colleges; completing goals; obtaining appropriate employment; tacking o f honors, awards, and achievements  •  Self-evaluation  .  Employer satisfaction  .  Student goal attainment and values inventories  •  Student perceptions o f non-cognitive outcomes  •  Student satisfaction, experience or opinion surveys (of program completers and noncompleters)  Other students: • Non-returning student survey (goal completion information)  Faculty: •  Faculty survey o f student engagement  Institutional Data In contrast to the first two indicators, which either directly or indirectly evaluate student learning, institutional data, also referred to as non-measures o f student learning, can be used for monitoring internal improvement o f the college (Green & Castelli, 2002; Walvoord, 2004). This type o f information is typically quantitative data and it may be compared longitudinally or may be used to monitor the progress o f an entire cohort o f students. Typically, colleges use institutional data to address important concerns or answer pertinent questions. Examples o f institutional data are listed below in Table 3.7 according to their data source (Alfred et al., 2000; Banta, 1999; C S H E , 2002; Green & Castelli, 2002; M a k i , 2002; J. Nichols, 2002; Seybert, 2003; Walvoord, 2004).  95  Table 3.7 Examples o f Institutional Data  Current students: • Grades . GPAs • Levels o f participation in the following: -> co-curricular activities -» learning communities -> community service / volunteerism Graduates: . GPAs . Grades (i.e., performance at receiving universities) . Graduation rates and numbers • Honours or other achievements by graduates • Job placement rates Faculty / staff/ administrators: • Faculty: -> faculty/student ratios -> faculty flow, retention and salaries -> faculty evaluations: peer review and chair/supervisor evaluation -> publications and recognitions . Administrator reviews/evaluations • Teaching and assessment methodologies survey • Staff reviews/evaluations  Enrollment services: • Attrition/retention rates (for all levels of courses): -> course level (completion rates) -» program level -> college level • Course-taking patterns and profiles . Demographic data / diversity o f student body (number o f students) • Enrollment trends / admissions practices • Grade distribution analysis • Length o f time to degree • Success rate o f developmental students in subsequent college-level courses Program: • Content analysis and review o f program, courses, syllabi and curricula . Cost • Enrollment patterns • Course-taking patterns • Degree completion rates . Number o f graduates . Student G P A s Institutional: • Comparison o f outcomes with peer institutions • Cost-benefit analysis (financial costs) • Curriculum / syllabus analysis (e.g., analysis o f transfer student preparation) • Strategic planning reviews Community: • Community perception o f program effectiveness • Responsiveness to community needs  96  Ideally, assessment at a college involves a combination o f approaches integrating multiple sources o f information from all three types o f data (i.e., direct indicators, indirect indicators, and institutional data).  A s s e s s m e n t Issues Specific to T w o - Y e a r Colleges a n d T r i b a l Colleges  Assessment can be particularly challenging in two-year colleges because o f their wideranging and numerous missions (Banta, 2004). Tribal colleges face many o f these same challenges. These colleges typically provide a wide range o f options: vocational programs, developmental courses, adult basic education and G E D courses, continuing community education courses, and, o f course, college-level courses and degrees for transfer to four-year institutions. A s a result, the nature o f the student body and faculty at community colleges is quite different from that o f those at four-year colleges — and both the students and instructors affect assessment. Students attend community college for a number o f reasons. Some students enroll at a college to complete adult basic education courses; other students enter college with the goal o f completing an associate's o f arts degree in order to transfer to a four-year institution; and others enter with the goal o f completing a two-year terminal or vocational degree in order to obtain employment. Still others enroll in one or two courses just for interest, with no intention o f ever completing a degree. A s a result, these students are more likely to appear to stop-out or dropout than students attending four-year colleges. Due to open admission policies, community college students also have a wider range o f academic abilities and preparation, which makes it difficult to conduct any cohort-based research that demonstrates value added. A l s o , because community college students tend to be more transient (sometimes as a result o f financial limitations), tracking them once they leave the college is a significant problem (Banta, 2004; Mundhenk,  97  2004). Because many community college students attend their classes and then leave campus and because it is more difficult to engage two-year college students in meaningful ways outside o f the classroom, assessments are generally embedded in their courses (Banta, 2004). Instructors at community colleges are more likely to be part-time, adjunct, and overwhelmed with teaching responsibilities. Furthermore, because o f their temporary nature, part-time and adjunct faculty are unlikely to attend faculty meetings or be involved i n discussions regarding assessment o f student learning, and, therefore, less likely to "buy-in" to the assessment program. Because o f their higher teaching load, faculty at community colleges are less likely to be involved in assessment research, but are more likely to be involved i n developing student learning outcomes and meaningful classroom assessment techniques. A s a result, most o f the assessment research at community colleges tends to be conducted by one person i n an assessment department and focuses mainly on information gathering for external reports and institutional surveys (Mundhenk, 2004). Due to limited resources, community colleges integrate a number o f approaches i n thenassessment efforts including combining standardized measures o f student learning with locally developed measures (Banta, 2004); examining student goals at entry and exit; reviewing student functional skills at entry and exit; exploring active student involvement in learning through administering student engagement surveys; following up on the grades and retention o f graduates who transfer to four-year institutions; asking local employers about their needs; and exploring course completion, retention, and graduation information (Banta, 2004). M a n y colleges start by simply setting realistic goals and limiting selection o f their learning outcomes to the three or four most important ones (Walvoord, 2004). Regardless o f the findings, the results o f each o f these  98  measures provide a point at which to initiate meaningful conversations among the members o f the college community. O n a positive note, the mission o f community colleges, that o f serving the local community's needs, lends itself well to the "public nature o f assessment" (Mundhenk, 2004, p. 38). Community colleges are i n the business o f providing comprehensive services to external bodies to which they are accountable, such as associate's transfer degrees to students who transfer to universities and two-year terminal technical degrees to students who seek employment in local businesses. Because community colleges serve the needs o f these external entities, their assessment efforts tend to reflect that focus. Thus community colleges frequently seek information about the level o f graduates' preparation by such means as surveying employers, surveying graduates, and tracking transfer students through transfer reports and communications with receiving four-year colleges. A s a result o f this external feedback, community colleges and their instructors adjust their programs, curricula, and teaching methodologies. I f done effectively, community colleges have the opportunity to transform themselves through this essential accountability process. It appears that assessment has been more successful, to date, in smaller colleges (Farmer, 1988; Mentkowski et al., 2000). This is probably because smaller colleges are generally more committed to excellence in teaching (as opposed to research) and are more able to cultivate a unified vision among faculty and staff, who can be involved directly in the development o f the mission, goals, outcomes, and assessment program. Developing an assessment program appears to be more difficult i n larger colleges where faculty frequently perceive the goals o f the college as conflicting with their own interests or even the interests o f their students. Despite these constraints and hurdles, more and more colleges, both large and small, are attempting to  99  implement assessment programs (Palomba & Banta, 1999). The need to satisfy accreditation requirements undoubtedly provides much o f the initial and ongoing motivation in most cases, as does the lure o f additional funding or the threat o f reduced funding (J. Nichols, 2002; Seybert, 2003).  E m e r g i n g I n n o v a t i v e A p p r o a c h e s to A s s e s s m e n t  Engagement One innovative approach to assessment i n two-year colleges is the use o f student engagement surveys, such as the Community College Survey o f Student Engagement — a standardized national survey that attempts to assess student motivation and involvement i n their learning process — or engagement. This survey solicits students' perceptions o f their level o f engagement in various aspects o f college life, such as collaborative learning, student effort, academic challenge, student-faculty interaction, and support for learners (Community College Survey o f Student Engagement, 2004). This information is valuable because research indicates that students who are more engaged in educationally purposeful activities like these and i n the college environment are more likely to succeed ( K u h , 2001; J. Nichols, 2002). Not surprisingly, preliminary research conducted i n this area found that the students who withdrew from college were the least engaged (R. Hughes & Pace, 2003). Colleges can use the results from this highly respected national survey to initiate meaningful conversations about increasing retention rates and improving student learning, as well as improving the quality o f the overall college experience. Chickering and Gamson (1987) refer to seven good practices in undergraduate education, which result in effective teaching and learning. These practices  100  encourage student-faculty contact; encourage cooperation among students; encourage active learning; give prompt feedback; emphasize the time the student devotes to the task; communicate high expectations; and, respect diverse talents and ways o f learning, (p. 15) Astin reaffirms and expands upon these practices by outlining three essential "conditions of excellence" for student learning: a high level o f meaningful and active student involvement in the learning process; high expectations by the students themselves and by the instructors; and high quality instructor assessment and feedback (Astin, 1996). The Community College Survey of Student Engagement is built upon this sound educational research, attempts to determine the level at which students are engaged i n the practices articulated by Chickering and Gamson and Astin, and provides a forum for discussing learning and teaching. In addition to the Community College Survey o f Student Engagement, there is a corresponding Faculty Survey o f Student Engagement, which focuses on faculty perceptions o f student engagement, the importance faculty place on different types o f learning and development, the frequency and nature o f faculty members' interactions with students, and how faculty members organize class time (Pike, 2003). The perceptions o f faculty and advisors toward each o f these areas can be compared with those o f students, which can inform conversations about teaching and learning and can lead to refinements in both. Because many o f the items included in these two surveys raise awareness o f issues and attitudes that faculty and advisors can recognize in conversations with students, the use o f these surveys has the potential to improve student retention (R. Hughes & Pace, 2003). B y utilizing these surveys periodically to assess student engagement, colleges can demonstrate improvement in the learning environment.  101  Assessment as Learning In the United States, a few colleges have taken the lead in engaging students in the assessment process itself— as a means o f promoting learning. Alverno College, a small Catholic liberal arts college for women i n Milwaukee, Wisconsin, is one o f these. Alverno College has been a leader in assessment for the last 30 years and has pioneered the concept o f assessment as learning (Mentkowski et al., 2000). Faculty and staff at Alverno College believe that students should learn something through every assessment process and, i f assessment is done well, the result should be learning that lasts (Mentkowski et al., 2000). Thus, assessment at Alverno College is as much a part o f the learning process as it is a part o f the evaluation process — and the success o f this approach has been impressive. Serving a predominantly low income, minority population, Alverno College (both its faculty and students) has received numerous awards and widespread recognition for its achievements (Alverno College, 2004). Mentkowski (2000) explains that this success is a result o f Alverno College's philosophy o f assessment, which is based on the following principles: 1. If learning is to be integrative and experiential, assessment must judge performance. 2. If learning is to be characterized by self-awareness, assessment must include expected outcomes, explicit public criteria, and student self-assessment. 3. If learning is to be active and interactive, assessment must include feedback and external perspectives as well as performance. 4. I f learning is to be developmental, assessment must be cumulative and expansive. 5. If learning is to be transferable, assessment must be multiple in mode and context, (p. 60)  These principles form the foundation o f all learning and teaching at Alverno College, as it pushes the concept o f authentic assessment farther than most other institutions. In the process, Alverno College established a standard o f experiential learning by integrating internships,  102  content, and abilities, and by defining learning outcomes as abilities. To support this approach, Alverno College's curriculum is ability-based: Students are required to demonstrate competence at increasingly advanced levels in eight abilities throughout the course o f their studies. Alverno's eight college-wide abilities are "communication, analysis, problem-solving, valuing i n decision making, social interaction, global perspectives, effective citizenship, and aesthetic responsiveness" (Mentkowski et al., 2000, p. 63). Because the assessment o f these eight abilities is integrated into a coherent educational system, students make connections across different disciplines (as well as within their major). Furthermore, Alverno College students are required to track the progress o f their own learning outcomes. Because the importance o f self-reflection is emphasized, students at Alverno become motivated, responsible, and accountable for their own learning and progress (Mentkowski et al., 2000). A t a workshop at Alverno i n June 2003,1 observed one o f the many intermediate-level ability-based assessments o f the communication ability. The process was simple yet impressive. Alverno staff placed students in teams of eight and gave students a sophisticated problem to solve, with an ethical dimension related to their field. Several retired professionals i n their field, from outside the college community, observed the students consulting for approximately one hour and provided them with detailed and immediate individual and group feedback according to specific criteria. The assessment also required students to provide a self-assessment and peer review o f their performance. It was obvious that by the end o f the afternoon, the students had gained significant and valuable insight into strengths and weaknesses o f their ability to communicate. M a n y academics and institutions have written about and supported some o f the principles listed above (Gipps & Stobart, 2003), but few have been as successful as Alverno College i n  103  implementing them i n such a comprehensive assessment system. For instance, Strain (2003) recommends that all assessment tools used i n the classroom should directly benefit students and their learning. The Center for the Study o f Higher Education (2002) suggests assessing learning outcomes at three points during the student's college experience: at admission (beginner level), at the midpoint o f their college degree (intermediate level), and at graduation (advanced level). They recommend that each program map out their curricula. For each learning outcome, they recommend outlining in a chart the level o f learning based on B l o o m ' s Taxonomy (i.e., knowledge, comprehension, application, analysis, synthesis, and evaluation). Tinto (1993) reaffirms the importance o f immediate feedback to effective individual student learning and Woosley et al. (2003) stress the importance o f putting larger scale structures i n place to respond immediately to students who go astray in their first year o f college. Furthermore, the National Postsecondary Education Cooperative (2002) affirms Alverno's competency-based approach in a report stating "strong competency-based initiatives produce meaningful assessment results that are used in making critical decisions about ways to improve student learning" (p. 19). Other aspects o f Alverno's approach that are referred to in a report include the importance o f the transferability and applicability o f knowledge and understanding, practice and feedback, and meta-cognition for self-correction (Pellegrino et al., 2001). This report notes that assessment cannot be kept isolated from curriculum and instruction — although it stops short o f stating that every assessment process should result in learning. The Committee on the Foundations o f Assessment (Pellegrino et al.), in an extensive report produced in 2001, used as the foundation o f its work two basic principles: (a) students should always learn something from being assessed and (b) this information should be used to improve student learning. Supporting Alverno's approach, the committee emphasizes that in  104  order for assessment to be successful and improve learning, students must understand the learning outcomes and their relevancy. It reflects on the importance o f integrally relating instruction with assessment as w e l l as timely feedback. The committee also recommends what Alverno has long known, that "assessment practices need to move beyond a focus on component skills and discrete bits o f knowledge to encompass the more complex aspects o f student achievement" (Pellegrino et al., 2001, p. 3). Alverno faculty address this concern by assessing their students while they are applying the knowledge and understanding they have gained through their education to other situations (Mentkowski et al., 2000). Thus, while reports such as this one are just beginning to articulate and recognize these important principles, Alverno has embodied them for the past 30 years.  Alverno's success in this regard is largely due to its  faculty's research on the assessment o f student learning. Regardless o f the professors' discipline, Alverno faculty are encouraged to engage in assessment-related research.  Planning and Reporting  The Assessment Plan In its Accreditation Handbook, the Northwest Commission on Colleges and Universities (2003), the accreditation body that oversees the colleges and universities in the northwestern United States, emphasizes the importance o f the college and program assessment plans: The Northwest Commission on Colleges and Universities expects each institution and program to adopt an assessment plan responsive to its mission and its needs. In so doing, the Commission urges the necessity o f a continuing process o f academic planning, the carrying out o f those plans, the assessment o f the outcomes, and the influencing o f the planning process by the assessment activities, (p. 37) The assessment plan serves either one or both o f the following purposes: first, to outline existing assessment strategies and to recommend ways to improve assessment; or, second, to  105  sirmmarize outcomes o f assessment measures and recommend ways to improve student learning (Walvoord, 2004, p. 9). If the purpose o f the assessment plan is to improve assessment, then the assessment report may include the following information: the person responsible for overseeing assessment; the resources and structures available for assessment; how faculty created the learning outcomes and how they are used for assessment; the measures o f student achievement, the reasons chose these measures, how they relate to the outcomes, and how they are administered; how assessment results are used for improvement o f learning; and recommendations to improve the assessment program (Walvoord, 2004). If the purpose o f the assessment plan is to improve student learning, then the assessment report should include a list of the learning outcomes; data from assessment measures and what the results suggest about student achievement o f the learning outcomes; and recommendations to improve student learning (Walvoord, 2004). N W I C ' s 2002 - 2006 assessment report primarily focuses on the first purpose, recommending ways to improve assessment. The 2007 to 2012 assessment report w i l l primarily focus on the second purpose, recommending ways to improve student learning (although it w i l l also recommend ways to improve assessment). The assessment plan coordinates campus-wide assessment efforts and college activities, presents to external bodies a well-conceived approach to assessment, and provides a systematic way to determine the extent to which outcomes have been achieved. The assessment plan focuses on educational outcomes and is dynamic i n nature (J. Nichols, 2002). Furthermore, the assessment plan may be a simple matrix that overviews the time-frame for accomplishment o f each o f the items and may include strategic plan, the college assessment plan, the academic program assessment plans, and a faculty evaluation process (Eastern Oregon University, 2004).  106  In order to build an assessment plan or report that is useful to the college, several steps must be taken: first, embed assessment in essential processes (such as strategic planning, curriculum review, program review, grading, developing independent learning programs or learning communities, and teaching evaluations); second, articulate college outcomes; third, conduct an assessment inventory (i.e., an inventory o f what the college is already doing or planning in terms o f assessment and how the information is used for making decisions and improving the college); fourth, highlight strengths and weaknesses and make recommendations for improvement; fifth, improve the assessment program; sixth, improve student learning; and, finally, write the plan or report (Walvoord, 2004).  Closing the Loop Assessment plans can be implemented at the course, program, and college levels. For instance, a basic assessment plan at the course level may include the instructors' learning outcomes; the tests and assignments; the instructors' criteria and standards (i.e., rubrics); student scores over time; and evidence o f feedback into learning and teaching (Walvoord & Anderson, 1998). A n example o f a section o f N W I C ' s assessment plan for 2007 - 2012 is included in Appendix A . Using a matrix, similar to the section o f the program-level plan in Table 3.8, provides a framework for encouraging faculty and staff to use the assessment plan and results to improve learning.  107  Table 3.8 Example o f a Section o f an Environmental Studies Program Assessment Plan Program outcome  Assessment criterion and procedure  Students w i l l be able to propose solutions to and solve realworld problems by applying the correct numerical data.  80% o f students completing the program w i l l score 4 or higher on the rubric for this outcome for the final project in the graduating seminar.  Assessment result  Use o f result  Only 70% o f students completing the program had a score o f 4 or higher on this rubric for the final project in the graduating seminar.  This concept w i l l be reinforced in two additional required environmental studies courses.  Note. Adapted from J. Nichols (2002, p. 247).  After the college implements its plan, it may report its results i n various ways. For instance, L o w e r Columbia College presents the results o f a different section o f its assessment plan in a reflective meeting every second month to its board, faculty, staff, and students (Weinstein, 2003). In this way, it can focus on a new area every couple o f months and present more manageable portions o f information to its community. Its assessment process includes a deep reflection on the results and recommendations for changes for the future. It also reports on results o f changes made in previous years. Other colleges opt for annual report cards that report on the progress o f institutional data (Banta, 1999). This option can be overwhelming as it requires a more substantial annual reporting project and is more o f a one-time effort. Other faculty and staff meet regularly through faculty retreats, meetings, forums, and so forth to reflect on experience gained through the assessment process and translate this learning into actionable reforms (Ewell, 2002). Regardless o f the reporting method chosen, because feedback o f the assessment results is a significant step i n the assessment process, it should not be rushed;  108  adequate time should be allowed for reflecting upon the information (Rodrigues, 2002). Furthermore, effective assessment requires that colleges move beyond merely documenting processes to evaluating the college's overall effectiveness (Banks & Colby, 1989).  Conclusion A well-planned and supported assessment program has the potential to lead to a transformed college environment and toward learning that lasts; however, transformation takes time and is influenced by the college's culture (Mentkowski et al., 2000). Thus, in order to create a culture o f assessment, assessment must permeate all aspects o f the college — from planning, budgeting, program mission statements, and the first-year experience program, to the catalogue, publications, website, job descriptions, and promotion. Just as self-assessment and reflection are essential for the development o f the student, so, too, are they necessary to sustaining a process o f institutional transformation, which w i l l expand awareness o f strengths and weaknesses in the quest to become a learning-centered institution (Mentkowski et al., 2000).  109  C H A P T E R 4 ASSESSMENT IN T R I B A L  C O L L E G E S  This chapter is divided into two sections. The first section discusses the guiding principles o f culturally appropriate research in Native American communities that are most relevant to the future o f assessment at tribal colleges. The second section provides a discussion of the state o f tribal college assessment i n the United States and an initial exploration o f possibilities for adapting best practices in mainstream assessment to a tribal college context in culturally appropriate ways.  Guiding Principles  The literature on research with Native American communities emphasizes the commitment o f researchers to a participatory process, based on a respectful and reciprocal relationship between the researcher and the community, when they conduct research in these communities (Haig-Brown & Archibald, 1996; Hampton, 1995b; Kirkness & Barnhardt, 1991; Menzies, 2001; National Sciences and Engineering Research Council o f Canada, 1999; Royal Commission on Aboriginal Peoples, 1996; L . Smith, 1999). In addition, the research itself needs to be relevant and meaningful to the community and conducted in a responsible and ethical manner (Darou et al., 2000; Greenwood & L e v i n , 1998; Kirkness & Barnhardt, 1991; Royal Commission, 1996; L . Smith, 1999; Stringer, 1999). Over the last century, researchers have conducted a substantial amount o f research i n Indigenous communities; however, repeatedly, researchers have ignored basic principles o f respect and ethics ( L . Smith, 1999). Furthermore, researchers have misinterpreted and excluded Native Americans from telling their stories ( L . Smith, 1999). A s a result, scholars o f Indigenous research and knowledge are now formally articulating principles that are essential to integrate  110  into research conducted in Indigenous contexts (Battiste & Henderson, 2000; Blanchard et al., 2000; Haig-Brown & Archibald, 1996; Hampton, 1995a; Kirkness & Barnhardt, 1991; Menzies, 2001; N S E R C , 1999; Royal Commission, 1996; G . Smith, 1992; L . Smith, 1999; Steinhauer, 2002; S. Wilson, 2003). Methods o f research appropriate in Native American communities are unique in that they "privilege(s) Indigenous concerns, Indigenous practices, and Indigenous participation" ( L . Smith, 1999, p. 107) by recognizing their distinctive worldviews and knowledge; honouring Aboriginal traditions and lands; "emphasizing the social, historical and political contexts, which shape" their lives; "privileging the voices, experiences and lives o f Aboriginal people and Aboriginal lands; and, identifying and redressing issues o f importance" to them (Martin, 2001, p. 3). This section reviews and elaborates upon the essential principles o f respect, responsibility, reciprocity, and relevance that apply most directly to the assessment processes at tribal colleges and to this research (Kirkness & Barnhardt, 1991).  Respect The importance o f researchers or, in this context, tribal college assessment coordinators maintaining respectful relations and a sense o f humility with those participating i n the assessment process is emphasized throughout the literature on Native American research (HaigB r o w n & Archibald, 1996; Menzies, 2001; N S E R C , 1999; R o y a l Commission, 1996; L . Smith, 1999). If the traditions, culture, and knowledge o f the tribal college community are protected and validated, the community becomes empowered. " A n essential and defining element o f ethical issues i n research on Indigenous peoples is respect for Indigenous peoples' laws and institutions" (Battiste & Henderson, 2000, p. 144). Taking it a step further, Linda Smith (1999) maintains not only that research "processes are expected to be respectful" but that, when carried  111  out properly, they can also "enable people, to heal and to educate" and "lead one small step further towards self-determination" (p. 128). Recognizing that this is the ideal and that it is a difficult process, Menzies (2001) also outlines four basic steps comprising a respectful research protocol. First, the researcher or the community initiates meaningful consultation in response to a college need or concern. Second, the researcher refines the research plan in consultation with the appropriate faculty, staff, students, or community members. The tribal college contributes a lot o f the input and directs the project's goals and outcomes. It ultimately determines i f the projects and methodologies being proposed are appropriate. Third, the research is conducted. Fourth, the researcher, always maintaining close contact with the tribal college community, writes, analyzes, revises, and distributes the report to the community. Involving many members o f the tribal college community in this way, Smith (1999) asserts, results in an effective process and a significant increase in the capacity o f the community. Needless to say, this approach often results in a prolonged research process and thus requires self-reflection, patience, flexibility, and openness on the part o f the researcher (Steinhauer, 2002). I attempt to follow these fundamental guidelines, both in my work and in my doctoral research, as I have responded to and been flexible with college needs; consulted continually with faculty, professional staff, and administrators; had numerous faculty, professional staff, and administrators review and edit my work; and reported results back to the tribal college community.  Responsibility M a n y o f the principles and thoughts that are articulated i n Indigenous research literature are also gaining momentum i n mainstream research literature and approaches. One o f these "approaches to participative inquiry" (Bensimon et al., 2004, p. 109), called Participatory A c t i o n  112  Research (also known as Community A c t i o n Research), seeks to maximize responsible research practices by engaging communities as full participants in the research process i n order to empower them (Greenwood & L e v i n , 1998; Stringer, 1999). A collaborative and communitybased approach "overtly engages the human relationships that are involved, is concerned with the style and manner o f communication among people, and purposefully includes all those affected by the research as active participants in the process" (Stringer, 1999, p. xx). B y forming partnerships and by using community-level resources ( A . W i l s o n & Cervero, 1997), these approaches give those who are being researched control over the process, enabling them to build local community skills and resources (Hudson & Taylor-Henley, 2001; Weber-Pillwax, 2001). Responsible research approaches include the use o f ethical research protocols and professional practices (Royal Commission, 1996; L . Smith, 1999). They also require an understanding o f tribal college protocols and a reflective and non-judgmental attitude (S. Wilson, 2003). A s L . Smith (1999) points out, "the processes o f consultation, collective meetings.. .and shared decision making" (p. 129) are important aspects o f this process. Thus, ultimately, the tribal college decides which research is pursued; who the research is for; the effect it w i l l have; who carries it out; how it is carried out; how it is determined the research is worthwhile; who owns the research; who w i l l benefit; and how the findings w i l l be shared (L. Smith, 2000).  Reciprocity "Practices o f reciprocity" involve "negotiating better research relationships" ( L . Smith, 1999, p. 136 and p. 119) and are skills that are fundamental in the pursuit o f respectful research processes with Native American communities. Research in a tribal college context recognizes the inter-connectedness and interrelatedness o f the physical, intellectual, spiritual, and emotional dimensions and involves the individual, family, and community (Graveline, 1998; Weber-  113  Pillwax, 1999; S. Wilson, 2003). Because this research is community-oriented and requires a holistic approach, the assessment coordinator is accountable to the tribal college community, rather than to his or her own pursuit o f knowledge (Cardinal, 2001; Steinhauer, 2002). Historically, Native American people have not benefited from research conducted i n their communities; therefore, the research should originate i n the tribal college community, serve its needs, and enrich its members' lives (Marker, 2004; Weber-Pillwax, 2001; Weiss, 1998). Reciprocity implies a collaborative research process, one that enables community members to participate in the planning, implementation, and analysis o f research results (Royal Commission, 1996). It requires everyone's contributing input and control equally and results i n "changing the power relationship i n research partnerships" (Blanchard et al., 2000, p. 10). It encourages researchers to listen, to hear, to attempt to understand, and only then to speak (Blanchard et al., 2000). It emphasizes the importance o f frequently reporting back to the community for accountability purposes so that the community can use the information learned for improvement ( L . Smith, 2000). It implies that the research questions being asked are relevant, valuable, and important to the community, and it implies a commitment to building relationships, which takes time. Furthermore, the Report o f the Royal Commission on Aboriginal Peoples (1996) suggests that "wherever possible, research should support the transfer of skills to individuals and increase the capacity o f the community to manage its own research" (p. 327).  Relevance The Royal Commission also suggests important ways for ensuring that research is relevant to the community. When the tribal college is involved in defining the problems to be examined, it insures their relevance to the improvement o f the community. A s a result, the  114  research engages the community directly in formulating solutions to problems it experiences. Ways this can be accomplished include involving the tribal college community members in the design o f the project and incorporating local practices with technical knowledge (Greenwood & Levin, 1998). In addition, involving the community in the review o f the findings and their reporting validates the content o f the research and further builds upon the principles o f relevance and reciprocity (Stringer, 1999). Research conducted in a tribal college needs to be useful to the community, to recognize and legitimize Indigenous knowledge and skills, and to be grounded 12  in the lives o f the students and community (Kirkness & Barnhardt, 1991; Nelson-Barber, LaFrance, Trumbull, & Aburto, 2005; S. Wilson, 2003). Thus, Darou, Kurtness, and H u m (2000) concur that research in Native American communities needs to display significant social value as well as adaptation o f the instruments to the culture. N W I C has developed a mechanism for protecting the research interests o f the college and the tribal communities it serves. Its Institutional Review Board reviews all research pertaining to the college and its tribal communities with the goal o f conducting effective and culturally acceptable research that benefits its communities. In a similar way, N W I C has a five-member Board o f Trustees that governs the college. The trustee members are appointed by the L u m m i Indian Business Council and are representative o f the tribes N W I C serves. The role o f the N W I C Board o f Trustees is to maintain accountability to the larger tribal communities. In an effort to do this, the president and administrative team meet monthly with the Board o f Trustees to share triumphs and to consult about challenges. The Board o f Trustees and the Institutional Review Board both work to assist in maintaining respectful, reciprocal, relevant, and responsible relations between N W I C ' s endeavours and the local communities it serves.  12  Refer to chapter 2 for a discussion about Indigenous Knowledge and Culture.  115  Research in tribal colleges has the potential to transform local communities (Blanchard et al., 2000; Nelson-Barber et al., 2005; L . Smith, 1999), but, to do so, it "requires a context that is consciously considered and purposefully incorporated into the research by the researcher" (Weber-Pillwax, 2001, p. 166). A s long as the above-mentioned protocols o f respect are followed, mainstream research methodologies can be integrated into those o f tribal colleges (Marker, 2004); however, ultimately an Indigenous research methodology, based on Indigenous epistemology, is fundamental to the creation o f Indigenous knowledge (Weber-Pillwax, 1999, 2001; S. Wilson, 2003). Blanchard et al. (2000) elaborate further on the importance o f research in Native American communities: " W e are interested in research and enthusiastic about its usefulness, but we want it to be inclusive o f our knowledge, values and inquiry perspective, and we want to be genuine partners in its production, distribution and application" (p. 3).  State o f Assessment i n T r i b a l Colleges  To date, tribal colleges unfortunately have found it difficult to gather even basic assessment data. For instance, quantitative data measuring the most basic conventional institutional data referred to earlier (e.g., retention and graduation rates, time to degree completion, transfer and employment rates) are often difficult to collect in tribal colleges due to inadequate technology and to a lack o f experienced human resources (Gagnon, 2007). Thus, the Carnegie Foundation (1997) asserts, "increased funding for staff dedicated to this research role is a necessary next step i n the growth o f the tribal colleges" (p. 94). Moreover, these conventional indicators alone do not provide a useful picture o f what matters most to tribal colleges (Boyer, 2003).  116  While conventional assessment frameworks clearly do not reflect the values or address the needs o f Native American communities, progressive assessment frameworks may reflect these values and address these needs, due to their more inclusive, relational, and process-oriented approaches (Haig-Brown & Archibald, 1996; Kirkness & Barnhardt, 1991; National Sciences and Engineering Research Council o f Canada, 1999; Royal Commission on Aboriginal Peoples, 1996; L . Smith, 1999). The more innovative assessment programs focused on student learning outcomes are gaining attention as valuable tools i n post-secondary education, yet they are not well developed in a Native American context (Carnegie, 1997). I f they are developed in Native American contexts, these progressive assessment frameworks may anticipate and address concerns that derive from critical educational theory regarding issues o f power and control in assessment and the political and economic context and motives surrounding assessment practices (A. Wilson & Cervero, 1997; Forester, 1993; L . Smith, 1999). Traditionally, mainstream assessment has not been characterized by either the principles articulated above or the methodologies their implementation requires. Rather, it has been driven somewhat narrowly by Euro-American definitions o f student success, definitions that have, to some extent, been imposed on tribal colleges in the past. In addition, as is the case o f mainstream colleges, tribal colleges are now being asked to demonstrate institutional effectiveness to accreditation bodies in more rigorous ways. A s there is a potential tension between outcomes-based assessment and Native American worldviews, some tribal college advocates initially were understandably concerned that Euro-American definitions o f student success would be imposed on tribal colleges even more than they have been in the past (Crazy B u l l , 1994); however, with the new emphasis on learning outcomes assessment, many tribal college advocates believe they now have an opportunity to re-defme their own measures o f  117  success and, therefore, their own curricular and pedagogical values, in more culturally appropriate ways (Boyer, 2003). Moreover, even though mainstream colleges are focusing their assessment efforts primarily on academic learning outcomes, tribal colleges believe that cultural outcomes are at least as important as cognitive, psychomotor, and affective outcomes, i f not more important (Astin, 1999). The broader mandate o f tribal colleges also provides them with an opportunity to widen the scope o f their assessment efforts by articulating and documenting community-wide outcomes in the tribal communities they serve. Indeed, tribal colleges emphasize community-wide outcomes, such as the "self-determination o f Native Peoples" ( N W I C , 2005b, p. 4), in their mission statements. These outcomes are difficult to measure, especially in two-year colleges. To approach assessment in a meaningful way, tribal colleges would need to track not only their ability to improve the lives o f students, but also their ability to improve conditions within tribal communities. In this context, the task o f assessment in tribal colleges "is to measure a college's ability to both serve a community and build community" (Boyer, 2003, p. 143). To date, Paul Boyer is the only person to articulate elements o f an assessment model in a tribal college context. He suggests that tribal college assessment programs address six levels o f accountability, which can satisfy the needs o f accreditation bodies and funding agencies while also providing a means for tribal colleges to monitor their progress, demonstrate their strengths, act on their weaknesses, and fulfill their mission statements. H e suggests that tribal colleges focus their assessment efforts on the following six aspects: mission statement, cultural outcomes, larger campus community, learning outcomes, wider tribal community, and communities beyond the reservation (Boyer, 2003). In this section, I elaborate upon and apply Boyer's framework by drawing on current best  118  practices in mainstream assessment and adapting them in culturally appropriate ways to a tribal college context.  M i s s i o n Statement Boyer's first area o f accountability concerns familiarity with, and support for, the institution's mission statement. A s mentioned earlier, tribal colleges were established with a two-fold mission: (a) to preserve traditional tribal culture and knowledge and (b) to provide certificates or associate's (two-year) degrees that provide Native American students with better employment opportunities and facilitate transfer to four-year colleges or universities (Farm, 2002). This two-fold mission is not well understood, is complex, and is challenging to implement, let alone measure. Boyer (2003) suggests that faculty, staff, students, and the larger tribal community can collectively examine the mission statement, question it, refine it, and engage in on-going dialogue about it, in order to increase the level o f familiarity with and support for it. "Appreciation o f the mission statement," he states, "encourages members o f a college community to look beyond a lack o f institutional resources and focus instead on the strength o f the college's vision" (p. 144). The mission statement and goals o f a tribal college should provide a clear understanding of the college's institutional intentions so that program outcomes can be directly developed from and linked to them (J. Nichols, 2002). The mission statement and institutional goals are generally articulated through a strategic planning initiative. Once articulated, the mission statement and goals provide guidance for administrative decisions, direction for each o f its programs, a framework within which academic units can assess and improve their programs, and a structure for overall assessment and improvement (J. Nichols, 2002). Therefore, the mission statement and goals o f the college need to be specific enough to provide clear direction for the  119  college and highlight its unique niche (Mentkowski et al., 2000). When the mission and goals are clarified, and assessment indicators are developed, students are more likely to "experience the curriculum as purposeful and connected" (Mentkowski et al., 2000, p. 322). It is therefore important that the indicators chosen to demonstrate student success reflect the college's mission and goals. W h e n this is the case, it also becomes apparent when the mission statement and goals no longer reflect the direction o f the college and need to be updated. In view o f the challenges tribal colleges face both i n fulfilling their ambitious missions and goals and i n meeting the needs o f the students they serve, a process whereby they are able to measure their success and make adjustments and refinements in the direction they are taking is vital. For example, in addition to preparing students to transfer to four-year colleges, many tribal colleges provide access to career training and retraining, developmental coursework, continuing education, and, most significantly, cultural enrichment. Moreover, tribal colleges serve a challenging student population, many o f whom live below the poverty line and have minimal academic preparation. Students represent a wide range o f ages and have various intentions for enrollment (Boyer, 1990). Therefore, agreeing on a set o f institutional goals, which addresses the needs o f a diverse student population, can be quite a challenge. The process o f ongoing reflection needed for refinement o f mission statements, goals, and indicators assists tribal colleges' efforts to address these challenges and more efficiently focus their resources on their students' needs.  Cultural Outcomes The second area pertains to how successful the tribal college is at supporting cultural outcomes, such as increasing students' sense o f cultural identity and increasing connection to their tribe and community (American Indian College Fund, 2002). Tribal colleges are only just  120  beginning to articulate these outcomes, let alone assess them (Marker, 2000). For instance, Turtle Mountain Community College, a tribal college in North Dakota, bases its curriculum on the seven Ojibwa teachings o f bravery, wisdom, love, respect, honesty, humility, and truth (Ortiz & Boyer, 2003b). The college developed a survey to examine its students' learning experiences and to determine how effectively faculty were integrating traditional culture and knowledge into the curriculum. The first time staff conducted the survey the results indicated that only a handful of instructors were basing the curriculum on the culture. These results stimulated a flurry o f open discussions about priorities and new approaches. After many discussions and in-service meetings, increased cultural awareness resulted in greater integration o f culture into the classroom. Turtle Mountain Community College is now trying to determine how students are applying this cultural knowledge outside the classroom (Yellow B i r d , 1998-1999). In the winter quarter o f 2003, N W I C took a similar approach and had similar results. It conducted a survey o f instructors to determine the teaching and assessment approaches they were using (see Appendix B , Survey of Teaching and Assessment Methods, for a copy o f the report). This survey also asked basic questions about writing, technology, and Native American cultural content i n courses taught that quarter. A t least 35% o f all courses reported having minimal to no cultural enrichment with Native American materials, and only 15% reported being primarily based on Native American materials. Twenty one percent o f courses used Native American speakers and approximately half the courses reported meeting the needs o f students by incorporating Native American texts, perspectives, films, writers, websites, music, and art. The discussion o f these results contributed to faculty enthusiasm for establishing clear standards, including cultural standards, for all N W I C programs.  121  Tohono O ' O d h a m Corrvmunity College has another innovative approach. It requires all staff and students to take the college's Native American language and history courses within their first year o f employment or attendance. Several faculty have reported enhanced understanding o f cultural issues and a more unified approach with this initiative (St. Clair, 2006). One o f the most challenging areas o f this kind o f assessment in a tribal college context is that o f first determining and then assessing cultural outcomes. This is one o f the areas where the least amount o f clarity exists as it is up to each individual college to determine for itself its desired cultural outcomes. In addition, assessing these outcomes can become extremely difficult when individuals from different tribes are represented and have different views about this sensitive subject (Gagnon, 2007). Alaskan Native educators have developed a set o f culturally responsive standards for students, educators, schools, curriculum, and community, which may well serve as a model for Indigenous colleges. These standards are articulated i n a document titled Alaska Standards for Culturally Responsive Schools (Assembly o f Alaska Native Educators, 1998). Finally, Demmert (2005) highlights six criteria he believes are essential for culturallybased Native American education programs: Culturally-based Native American education programs (a) require students to learn Native languages; (b) have "pedagogies that stress traditional cultural characteristics"; (c) have teaching strategies that "are congruent with the traditional culture as well as contemporary ways o f knowing and learning"; (d) have a "curriculum that is based on traditional cultures, which recognizes the importance o f Native spirituality and places"; (e) "have strong Native community participation including partnering with parents, elders, and other community resources"; and (f) have "knowledge and use o f the social and political mores o f the community" (p. 4). When articulating cultural outcomes, tribal  122  colleges may want to consider Demmert's suggestions and the Alaska Standards for  Culturally  Responsive Schools (Assembly o f Alaska Native Educators, 1998).  Learning Outcomes Learning outcomes pertain to measuring student learning, including cognitive, affective, and psychomotor outcomes (I discussed cultural outcomes i n the Cultural Outcomes section, in chapter 4) (American Indian College Fund, 2002; Archibald & Selkirk, 1995). A l l learning outcomes may be demonstrated at three levels: the college level, where students are assessed throughout their time at the college with regard to a set o f general education or college outcomes; the program level, where students demonstrate their learning o f program outcomes through, for example, capstone experiences; and the course level, where students are evaluated on their shortterm learning o f course outcomes. A n example o f a common college outcome may be that students will be able to write Standard English.  A program outcome might be that students will  be able to conduct environmental assessments, impact monitoring, modeling, and prediction.  A  course outcome i n a math course might be that students will be able to complete an income tax return.  College Outcomes Tribal colleges are just beginning to articulate college outcomes. When students are evaluated for college outcomes both at entrance to and at exit from college, this is referred to as value-added assessment. Value-added assessment, where students are their own controls, emphasizes gains in students' abilities to identify the impact o f their college education. This approach requires buy-in from faculty and staff, effective leadership, an effective computerized tracking system, a commitment to use the results, and time (Stufflebeam, 2001). Experts in the  123  assessment field advocate the use o f course-embedded assessments (also known as secondary analysis o f course papers or projects) rather than, for example, the use o f separate "testing days" to assess college outcomes (Banta, 2003; Suskie, 2003). A s the name implies, course-embedded assessments involve evaluation o f student work that is done in the classroom; in this case, however, samples o f work may be assessed by faculty a second time to assess college outcomes. Course-embedded assessment becomes even more essential in tribal colleges where many students enroll with the intention o f only completing one or two courses (J. Nichols, 2002). The advantages o f this approach over separate testing days for a tribal college are that it is costeffective (because the same assessment tool can be used for two purposes) and some students are motivated by grades to perform their best. In addition, in the opinion o f Gagnon (2007), a professor o f Indian Studies at the University o f North Dakota and a tribal college evaluator, more innovative assessment strategies that take advantage o f small class sizes may be employed at tribal colleges: Small size means that assessment can be more informal — less testing instrument oriented. I f you have only 5 faculty members teaching general ed. courses, then they can get together regularly to describe student learning based on their intimate knowledge o f students in their courses and in the context o f the college, (p. 1)  Tribal colleges are beginning to think about college-wide assessment in more creative ways.  Program Outcomes The assessment o f program outcomes is central to a tribal college's commitment to accomplishing its mission and goals (J. Nichols, 2002; Rossi, Lipsey, & Freeman, 2004). Because implementation o f the college's mission takes place at the program level through the identification o f outcomes linked to the college's mission and goals, assessment results indicating that program outcomes are being accomplished imply that the college's mission is 124  being accomplished (J. Nichols, 2002). Tribal colleges are just beginning to develop programlevel assessment processes (Gagnon, 2007). Once an institution has agreed upon its mission statement and goals, there is general consensus among assessment experts about the subsequent steps that should be taken in developing a program assessment plan ( M a k i , 2002; Mentkowski et al., 2000; Rossi et al., 2004; Stark & Lattuca, 1997). The first step is for each program to identify and define its learning outcomes. These outcomes clearly state what the program would like its students to be able to do upon graduation, as well as specify the expected level o f performance. Next, the related activities and capstone experiences that w i l l be used to teach the material are identified, as well as appropriate evaluation methods (both quantitative and qualitative) for assessing student achievement for each outcome at key stages in the program. The number o f outcomes to be assessed (perhaps three or four) is limited to those that are generally weak and reasonably attainable, both "measurable" and observable, in the allocated period o f time (The State University o f N e w Y o r k Geneseo, 2004; Walvoord & Anderson, 1998). After determining how the results w i l l be used for program improvement, the faculty can then implement the evaluation methods to determine the extent to which learning outcomes are being achieved. In order to do this, criteria are identified that clearly articulate successful accomplishment o f learning outcomes (e.g., 80% o f students completing the program w i l l score 4 or higher on the rubric for this outcome for the final project in the graduating seminar) (refer to Table 3.8). Later, results are communicated to faculty, staff, and students so that recommendations and decisions can be made to improve learning. Ideally, the tribal college implements its assessment activities on a regular cycle during which it may review and possibly modify its mission, goals, or program outcomes. Ultimately,  125  of course, the results are used to improve learning and teaching. A s expected, a time-line is developed to track the assessment o f program outcomes, and the entire assessment process is reviewed to determine whether it is providing the most useful information possible (Bresciani, 2003; Hatfield, 1997; The State University o f New Y o r k Geneseo, 2004; University o f Wisconsin, 2004). T o assist in organizing this information, the process o f systematically determining which outcomes have been achieved and how the information w i l l be used can be recorded in a program assessment plan similar to the one i n Table 3.8. A point that is worth emphasizing is the order i n which assessment activities at the program level are implemented: The intended educational outcomes are established first and then the appropriate means o f assessment'are identified (J. Nichols & Nichols, 2000). The approach to assessment planning outlined above provides a valuable tool as it requires faculty and administrators to "close the loop" and to demonstrate the use o f assessment results to improve learning (J. Nichols & Nichols, 2000). James Nichols (2002) points out additional considerations that are equally important to the organization o f a successful program assessment plan. First, it is important to gain the confidence o f program chairs, faculty, and staff by carefully explaining the process and how the data w i l l be used. A s discussed in chapter 3, the key to the success o f any assessment program is that it is faculty-driven, so the administrators' roles are limited to encouraging and providing support and resources. Second, to ease the burden on faculty, who commonly lack the time and resources to be adequately engaged with the development o f an assessment plan, existing data and course-embedded assessments should be used as much as possible (J. Nichols, 2002). Third, successful program assessment plans focus their efforts on end results (outcomes), rather than evaluating the process, which has been the traditional focus o f assessment efforts at this level  126  ( S U N Y G , 2004). This outcomes-based approach w i l l be unfamiliar and even uncomfortable to some faculty as the focus is on the impact o f faculty actions (i.e., learning), rather than on the action itself (i.e., teaching). Finally, a successful program-level assessment process educates the students about the role o f assessment in their education (University o f Wisconsin, 2004). These different elements o f a successful program-level assessment process can be incorporated into a wide variety o f situations. Because tribal colleges provide an assortment o f programs to serve students with various educational objectives or intentions (e.g., two-year college transfer, continuing education, vocational training), the methods chosen to assess different programs might vary. Nevertheless, it is wise for all programs to select a combination of direct and indirect indicators o f student learning, as well as institutional data. For instance, when assessing a program that provides vocational training, a tribal college might collect information about how students performed in their apprenticeships or internship (direct indicator), how graduates evaluate their career training through focus groups or surveys (indirect indicators), how successful students have been in obtaining suitable employment (institutional data), or whether the college is meeting local tribal or regional labor market and economic needs (institutional data) (Alfred et al., 2000; Banta, 1999; Seybert, 2003). Likewise, a developmental education program at a tribal college may evaluate a writing assignment (direct indicator), conduct a course evaluation (indirect indicator), document course completion rates (institutional data), or monitor student migration from non-credit to credit courses and programs (institutional data) (Banta, 1999). Other forms o f program evaluation include external exam, portfolio, classroom, capstone course, research or senior thesis assessment (direct indicators); student selfassessment and entry, midway and exit surveys, focus groups or interviews (indirect indicators). Finally, as a side note, at the program level, although only a few relevant learning outcomes are  127  generally assessed, they are likely to be assessed in different courses at different stages o f program completion.  Course Outcomes A s I discussed in the previous section, program assessment combines the assessment o f key learning outcomes i n capstone experiences (direct indicators) with indirect indicators, and institutional data and reports the information back to the department in relation to program goals to improve learning and teaching. In contrast, i n course assessment, the work o f students is primarily evaluated by the instructors, using mostly direct indicators that are based on course outcomes to generate grades and to improve their teaching and the students' learning (Walvoord, 2004). James Nichols (2002) notes that learning outcomes are "changes that result from instruction" and must be "focused on student learning and the improvement o f teaching and learning" (p. 157). These learning outcomes may include those required to be successful in upper level courses, i n obtaining employment, or i n becoming a contributing member o f society (J. Nichols, 2002). It is preferable to assess these learning outcomes i n the classroom using authentic assessment tools. A n example o f an indirect indicator o f student learning at the course level is the use o f student course evaluations that include questions about how well students think they have mastered each o f the learning outcomes as well as the quality o f the instruction (Walvoord, 2004).  Authentic Assessment Techniques In a tribal college context, elements o f Indigenous knowledge can best be passed on to students when traditional teaching and assessment methods, such as apprenticeships, observations, ceremonies, and practice are employed (Battiste & Henderson, 2000). Crazy B u l l  128  (2006b) emphasizes that "Native people are experts in experiential learning, learning by observation and practice, and in reflective learning. Our cultural teachings are grounded i n this type o f learning" (p. 4). The incorporation o f traditional culture and knowledge throughout the curriculum makes the curriculum more relevant and accessible to students (Boyer, 1990; Fixico, 2000). Because students respond more favorably to experientially grounded and contextualized learning experiences, the use o f authentic assessment tools over traditional tests is a natural and more appropriate approach to assessment i n tribal colleges. Kirkness and Barnhardt (1991) emphasize the importance o f measuring students' achievement through the students' ability to effectively perform meaningful and contextually appropriate tasks.. .that allow for the integration o f various forms o f knowledge and the application and display o f that knowledge in a variety o f ways. (p. 10) N W I C , as an example, has experimented with assessing learning outcomes in more authentic and creative ways. Its Tribal Environmental and Natural Resources Management Program "emphasized practical competency" and "an ability to synthesize and to solve complex real-world problems": Important measures o f success include whether the students stay grounded i n cultural values; maintain self-respect and a healthy sense o f self within the context o f community; are able to use critical and integrative abilities to problem solve and imagine creative solutions; and make substantial improvement in writing, reading comprehension, speaking and computational abilities. (Burns et al., 2002, pp. 57-58) B y making the curriculum, as well as the assessment o f student learning, more relevant and authentic, students are able to make meaningful connections between their coursework and their life experiences. A s instructors in tribal colleges are being encouraged to increase the relevancy o f the coursework to their students' lives and to use more authentic assessment methods in class, portfolios are becoming a more widely accepted and encouraged tool for documenting student  129  learning and improvement over time. They provide a more creative and real-life option than traditional tests and are an effective way o f evaluating general knowledge and learning i n tribal college programs. Student portfolios are a collection o f the students' work over the course o f their college experience and may include writing samples, self-assessments and reflections, videotapes, speeches, and so on. Colleges that have introduced portfolio systems have found that portfolios encourage better communication between faculty, students, and advisors (J. Nichols, 2002). Student portfolios also encourage the use o f multiple assessment methods to evaluate competency o f important learning outcomes ( M a k i , 2002; S U N Y G , 2004). Ideally, several faculty should evaluate student work at multiple points o f their college career (Walvoord, 2004). Tribal colleges are already experimenting with promising assessment methods, such as portfolios, self-evaluation, and applied "real-life" situations, which can demonstrate student progress in the more elusive affective learning outcomes.  Larger Campus Community Boyer's third area o f accountability pertains to the larger campus community, including direct indicators (specifically college outcomes), indirect indicators, and institutional data, which I previously discussed in this doctoral thesis. Because tribal colleges' resources are substantially limited, it is highly recommended that they begin the assessment processes by using data that are currently available and taking an inventory o f their assessment practices i n a systematic way (i.e., an assessment inventory) ( C S H E , 2002; J. Nichols, 2002). M a n y tribal colleges are already doing some assessment work and have existing data — either direct or indirect indicators, or institutional data — that can be tapped. For instance, some have entrance or licensure exams, some test graduating students (with standard or localized tests or performance tests), and others conduct graduate or satisfaction surveys. In addition, most  130  colleges have an institutional data system (e.g., financial aid, registration, and payroll) from which they generate reports that can also be used for assessment purposes (e.g., transcript analysis). For example, all accredited tribal colleges are expected to generate Integrated Postsecondary Education Data System (IPEDs) reports, which include information about students, degree programs, faculty, and finances, and these reports may be adapted for use in the assessment program.  Direct Indicators (College Outcomes) Some tribal colleges are beginning to articulate broad college outcomes, such as writing proficiency, through entrance tests and then capstone courses, videotapes o f performances, portfolios, internships, or exit exams. M u c h work remains to be done i n this regard as this work is grueling and time-consuming and gets at the core values o f the college. Refer to the section titled College Outcomes i n chapter 4 for a more in-depth discussion o f college outcomes.  Indirect Indicators After a tribal college inventories its current assessment procedures, James Nichols (2002) recommends conducting attitudinal surveys. There are many types o f surveys that can be administered to current students, alumni, employers, faculty, staff, administrators, and the tribal community, depending on the needs o f the college. Colleges can either use existing standardized surveys or, depending on the resources available, develop their own. Fortunately, many standardized surveys now allow tribal colleges to add a set o f their own questions at the end o f the survey, which provides for customizability. Conducting national attitudinal surveys is one way to be active i n the assessment process, the advantages being the costs are minimal and the tests provide baseline information while the assessment program is being developed. Especially  131  in a tribal college context, alumni surveys can provide valuable information about further educational progress and performance, satisfaction with the college experience, and job acquisition and performance (J. Nichols, 2002). Finally, because a significant factor in student retention and success is satisfaction, these surveys can also provide useful information for assessment (Banta, 1999; J. Nichols, 2002; Ouimet, 2003).  Institutional Data Most tribal colleges are now tracking patterns o f student enrollment, retention, graduation, transfer, and, sometimes, employment. Although it would be extremely valuable for tribal colleges to collect, use, and interpret this information, several constraints exist (over and above lack o f finances and resources). One o f these is the challenge o f creating standardized definitions for calculating retention, graduation, and course completion rates. U p until recently, it was virtually impossible to compare data among tribal colleges, or from year to year within tribal colleges, because o f the lack o f consistent definitions. Tribal colleges are starting to define these rates. In fact, i n 2004, the American Indian Higher Education Consortium initiated a new tribal college project — called the American Indian Measures for Success: The American Indian Measures for Success ( A I M S ) project is a two year project with two goals: to define relevant quantitative and qualitative indicator data o f American Indian student success, as determined by the tribal colleges, and to develop and implement a strategy for collecting, analyzing, and presenting annually the success indicator data using electronic information management tools. Through data collection and analysis, this project w i l l be the foundation for systemic reform that significantly increases — and for the first time, accurately measures — American Indian success in higher education. (Systemic Research, 2006, p. 1) This comprehensive and uniform set o f tribal college data is now being collected annually from all tribal colleges and is being used by A I H E C to appeal for tribal college funding and support.  132  One constraint o f collecting assessment information is that important data, which are needed to interpret assessment results, are frequently missing. For example, knowing a student's intentions (e.g., to obtain a two-year degree, to learn a traditional language or culture) upon enrollment is central to any assessment effort — especially i n a tribal college setting — because it is difficult to monitor student success without it. For instance, in order to calculate a graduation rate, the college must only include i n the calculation those students who enrolled with the intention o f completing a two-year degree (J. Nichols, 2002). The college can use this information for many purposes. B y combining the students' intentions or reasons for attending college with other information such as G P A , gender, age, or number o f children, important information can be generated about patterns o f student flow. Moreover, tribal colleges can determine how successful students are at accomplishing their goals and explore whether students upgrade their original goals after attending college (J. Nichols, 2002). The information gleaned from multiple assessment activities at the larger campus community level should include direct and indirect indicators and institutional data and can be used extensively for many purposes, including initiating meaningful conversations about improvement among faculty, staff, students, and the community.  Wider Tribal Community The next level o f accountability Boyer refers to is to the wider tribal community — the "importance o f connections, service, and community" (Ortiz & Boyer, 2003a, p. 46). Toward this end, tribal colleges are exploring what it means to be agents o f change in the larger community, as well as how to measure the impact o f the college on the community. Eventually, tribal colleges hope to strengthen local economies, improve standards o f living, increase household income, revitalize Native language and culture, restore traditional values and skills,  133  rebuild cultural knowledge, and generally provide hope for tribal community members (Boyer, 2003). For example, Turtle Mountain Community College is making an initial attempt to measure its impact on the wider tribal community; all staff (including faculty) are asked to participate in extended community affairs, and information about their staffs involvement in these community activities is reported in their final assessment report (Yellow B i r d , 1998-1999). Given their limited resources, it is extremely difficult for tribal colleges to assess their impact on the wider tribal community. Community impact surveys o f local tribal members can enquire about the amount o f contact they have with the college as well as their overall impressions and satisfaction with its programs and services (Alfred et al., 2000). In addition, tribal colleges could benefit from needs assessments to determine the priorities from the tribal members' perspectives. Furthermore, alumni surveys can provide much-needed information regarding the ultimate economic, personal, cultural, and other benefits for the students and communities served by tribal colleges. Finally, Crazy B u l l (2004) notes that the trend toward incorporating service learning at tribal colleges has provided a formal mechanism for student and community engagement. Service learning experiences and learning communities help colleges and universities address key elements o f accreditation — student outcomes, assessment, and community engagement... Success o f students — their retention and completion o f their programs o f study are closely linked to their engagement — so these opportunities also strengthen the most basic mission o f higher education in the United States — to develop an educated, pluralistic, democratic population, (p. 1) President Crazy B u l l explains that despite over 30 years o f control o f Indian Education by Native people, we still struggle to create culturally-relevant opportunities. Tribal colleges that have been using service learning strategies find that those opportunities are very grounded in community and that they can make a difference in their communities, (p. 1)  134  A s expected, a great deal o f work needs to be done in assessing the impact o f tribal colleges on their wider tribal communities and in creating culturally relevant learning opportunities. Nonetheless, the trend toward formally incorporating service learning into tribal college curricula is making that job easier. Finally, i n order to assess the impact o f tribal colleges on local communities, tribal colleges may develop direct indicators, indirect indicators, and data at the community level to monitor their progress.  Communities beyond the Reservation Boyer's final level o f accountability pertains to communities beyond the reservation, including accreditation bodies, federal funding agencies, and charitable organizations who are supporting the tribal colleges. Although positive in many ways, these relationships sometimes result in tension when tribal colleges' values are questioned or challenged. For instance, Crazy B u l l (2006) states that for tribal colleges, accreditation means "participating i n a process that requires us to integrate our understanding o f the world as Native people and as Native educators with a different, often western/European based understanding o f higher education" (p. 6). She outlines special accreditation issues for tribal colleges including the unique cultural environment, language experiences, social and familial relationships, tribal government's role, rural and often economically poor and isolated environments, older student population, increasing need to respond to younger students with different expectations, majority o f students are women, unusually high number o f non-college ready students, generally very limited financial and human resources compared to other educational institutions, often different delivery strategies in use, and pedagogy o f teaching and learning based in tribal knowledge and worldview. (p. 7)  135  However, as long as tribal colleges are dependent upon, and accountable to, these agencies for accreditation, funding, or support, tribal colleges w i l l have to use assessment in creative ways to resolve this tension. W o r k in this area has barely begun. With the limited time and scarce resources at their disposal, tribal colleges struggle with balancing the needs o f the accreditation bodies and funding agencies with the priorities and values o f the college, as well as their capacity to generate the data. In addition to the overwhelming challenge this presents, Marker (2000) points out a potential challenge in the accreditation process: " A s First Nations continue to create programs that celebrate and promote language and identity, they must negotiate cultural outcomes with agencies and institutions that control funding and accreditation" (p. 30). To address this difficulty, tribal colleges might ultimately have to play a role in educating the accreditation bodies and funding agencies regarding culturally appropriate outcomes. If tribal colleges are successful, the accreditation bodies and funding agencies might be more receptive to the colleges' "culturally responsive structures and programs" (Marker, 2000, p. 30) and responsive to the creative, unique, and unconventional ways tribal colleges choose to assess their cultural outcomes. The challenges for colleges w i l l be to determine how to assess such outcomes in a culturally appropriate manner. On the other hand, this challenge presents an opportunity to break new ground, not only in the field o f tribal college assessment, but also in the entire assessment field — which has traditionally ignored these kinds o f outcomes. In the meantime, the W o r l d Indigenous Nations Higher Education Consortium ( W I N H E C ) formed in 2002 with the intention o f creating an accreditation process for Indigenous higher education institutions worldwide. It is "committed to building partnerships that restore and retain Indigenous spirituality, cultures and languages, homelands, social systems, economic  136  systems and self determination" and "united i n the collective synergy o f self determination through control o f higher education" ( W T N H E C , 2003, p. 3). Although unique i n its mission in that it recognizes Indigenous culture, traditions, worldviews, and history as valuable assets, like mainstream accreditation bodies, W L N H E C evaluates the quality o f a college by how well it is able to accomplish its mission ( W I N H E C , 2003). W I N H E C membership includes individuals from institutions from Alaska, Aotearoa (New Zealand), Australia, Canada, Hawaii, Saamiland (Norway) and the continental United States. To date, W I N H E C has approved three undergraduate programs — from Te Wananga o Aotearoa, Te Whare Wananga o Awanuiarangi, and Te Wananga-o-Raukawa — all colleges in N e w Zealand. In 2006, it supported an accreditation request from the Seven Generations Education Institute o f Ontario and Sami allaskuvla/Sami University (Norway). Other institutions intending to pursue W I N H E C accreditation include the First Nations Technical Institute (Tyendinaga M o h a w k Territory, Canada); K e A l a Pono (an Indigenous college i n Hawaii); Halau Wanana and K a Haka 'ula o K e ' e l i k o l a n i (at the University o f Hawaii); and the University College o f the North Manitoba (Canada) ( W I N H E C , 2006, p. 13).  Conclusion Using a holistic approach, the first half o f this chapter proposed a set o f ethical considerations that need to permeate assessment processes at tribal colleges with the belief that the assessment process itself is as important as the ultimate product. Linda Smith (1999) points out that "negotiating and transforming institutional practices and research frameworks is as significant as the carrying out o f actual research programs" (p. 140). This w i l l be evidenced when the work proceeds in a respectful, responsible, and reciprocal manner, where relationships are developed through a participatory and consultative process, and when the project can  137  maintain its relevance for the college and all those involved. The Northwest Commission on Colleges and Universities (2005) reaffirms this belief when it emphasizes that personnel from all segments o f the institution, including faculty, staff, students, administration, and the governing board, must be included in the accreditation, self-study, and assessment processes. This participatory approach is important because processes o f consultation and shared decision making are essential aspects o f educational research i n a tribal college context ( L . Smith, 1999). The second half o f this chapter discussed current assessment initiatives in tribal colleges in the United States and explored possibilities for adapting best practices i n mainstream assessment to a tribal college context in culturally appropriate ways. I used Boyer's (2003) suggestion o f focusing tribal college assessment efforts on six levels o f accountability — the mission statement, cultural outcomes, larger campus community, learning outcomes, wider tribal community, and communities beyond the reservation — as a framework to explore these assessment initiatives.  138  C H A P T E R 5  M E T H O D O L O G Y  In order to respond to the research questions — (1) What criteria are best used to evaluate an assessment program in a tribal college context? (2) W h i c h elements o f the N W I C assessment program are most and least successful according to the evaluative criteria established in Research Question 1? and (3) What preconditions and other contextual factors contribute to the relative success or failure o f different elements o f the N W I C assessment program? — I conducted a case study o f the assessment program at N W I C . This approach honours the principles outlined by scholars o f Indigenous research in chapter 4, uses several data collection methods, such as surveys, participant-observation, and document review, and is informed by the Context-InputProcess-Product (CIPP) M o d e l developed by Daniel Stufflebeam (2003b). The case study approach and C I P P model are described below in more detail.  Case Studies  According to Robert Y i n (2003), a case study is an empirical inquiry that investigates a contemporary phenomenon within its real-life context, especially when the boundaries between phenomenon and context are not clearly evident.. .The case study inquiry copes with the technically distinctive situation in which there w i l l be many more variables o f interest than data points, and as one result, relies on multiple sources o f evidence, with data needing to converge in a triangulating fashion, and as another result, benefits from the prior development o f theoretical propositions to guide data collection and analysis, (p. 13-14) We can learn a lot from a single case that can apply to other cases and situations (Stake, 1995). A case study is appropriate under a number o f circumstances: when it is a critical case used to test existing theory; when it is a unique case; when it is a representative or typical case; when it is a revelatory case; and when it is a longitudinal case ( Y i n , 2003). In addition, a case may be used as a pilot.  139  This case study — the assessment program at N W I C — fulfills several o f these conditions. The study is representative,  in that N W I C is a typical tribal college, and it is  longitudinal, in that N W I C is easily accessible to me over a long period o f time. It is also a revelatory case in that it can reveal crucial information about assessment i n a tribal college context from which other tribal colleges can learn (by modifying generalizations about mainstream assessment to a tribal college context). Finally, it is a pilot, i n that it is one o f the first studies o f assessment in a tribal college context. The literature outlines various limitations o f case studies. In fact, some researchers call case studies weak, subjective, imprecise, and lacking rigor (Stufflebeam, 2001; Y i n , 2003). Y i n (2003), who has written a comprehensive text about case studies, counters these arguments by suggesting that these criticisms o f the case study method are "misdirected" (p. 10). H e believes the lack o f rigor often characterizing these studies has been mainly due to careless research techniques because skills for conducting case studies have not been well defined up until now. Stake (1995), another case study researcher, sees the subjective and qualitative nature o f case studies as a strength that is essential to understanding (as long as precautions are taken to minimize misunderstanding). A l s o , i n the past, case studies commonly took a long time to complete, were expensive, had little impact on social practice, and resulted i n long, poorly written reports (Stake, 1995; Y i n , 2003). A l l o f these concerns can be addressed i f the researcher takes the time to properly organize the case study. Others believe that case studies leave little grounds for generalization; however, Y i n (2003) claims that, although case studies cannot be used for "statistical generalization" (i.e., case studies typically do not collect empirical data that lend themselves to statistical analysis, so this form o f generalization is not typically made from case studies), they can be used to "generalize theories" (also referred to as "analytic  140  generalization") (pp. 31-32). I w i l l discuss the generalizability o f case study findings in more detail later i n this chapter. Case studies are commonly used because they are valuable tools, especially in evaluation research, that offer great flexibility to examine a program in an in-depth, holistic way as it naturally occurs ( A l k i n & Christie, 2004; Feagin, Orum, & Sjoberg, 1991; Merriam, 1998; Schwandt, 1994; Stufflebeam, 2001; Y i n , 2003). A case study can be used in evaluation research, such as this study, where it is used to explain, describe, illustrate, and explore various elements o f the N W I C assessment program and how these may apply to different situations (J. G a l l , G a l l , & Borg, 2005; M . Gall, G a l l , & Borg, 2003; Y i n , 2003). Stufflebeam (2001) evaluated 22 different evaluation approaches using the program evaluation standards o f utility, feasibility, propriety, and accuracy and found that the case study method was one o f the most effective. These program evaluation standards (The Joint Committee on Standards for Educational Evaluation, 1994) — some o f which have already been discussed in various sections of this doctoral thesis — are abided by during my research. The accuracy standard comprises reliability and validity issues, which are o f particular concern and which are elaborated upon below.  Validity and Reliability Issues in Case Study Design The quality o f a case study design is essential, and it can be evaluated using several validity and reliability criteria. Whereas validity is "the extent to which a measure actually measures what it is intended to measure" (Rossi et al., 2004, p. 436), reliability is "the extent to which a measure produces the same results when used repeatedly to measure the same thing" (Rossi et al., 2004, p. 433) The following section overviews three different types o f validity and one type o f reliability criterion and suggests ways o f improving the quality o f a case study i n  141  each o f these respects. In this case study, I attempted to maximize validity and reliability by addressing all o f these criteria.  Construct Validity Construct validity is the process o f "establishing correct operational measures for the concepts being studied" ( Y i n , 2003, p. 34). This is a challenge i n case study research because o f the subjective nature o f data collection (Denzin & Lincoln, 1994; Patton, 1987); therefore, i n order to minimize this concern, the researcher uses multiple sources o f information and "establish[es] a chain o f evidence" ( Y i n , 2003, p. 34) during the data collection phase. A "chain o f evidence" is a clear association between the research questions, the information, and the conclusions ( Y i n , 2003). The rationale for using multiple sources o f evidence.. .allows an investigator to address a broader range o f historical, attitudinal, and behavioral issues. However, the most important advantage presented by using multiple sources o f evidence is the development o f converging lines o f inquiry, a process o f triangulation. (p. 98) Thus, triangulation can be used to "disconfirm major assertions" (Stake, 1995, p. 87). A l s o , in order to improve construct validity, during the writing phase, the researcher has drafts o f the report reviewed by participants in the case study. This way, facts and interpretations can be corroborated and an opportunity is provided to make corrections ( Y i n , 2003). I used a combination o f complementary data collection methods or sources o f information — including surveys, participant-observation, and document review — during this case study to triangulate the data in an attempt to maximize construct validity (Stake, 1995). In this case study, I used at least one o f these three methods to gather the information needed to answer each o f the three research questions. I used the participant-observation method for each question. To illustrate: I used a review o f the literature combined with my observations and  142  experiences in the assessment field at a tribal college as a basis for the response to Research Question 1. In addition to these two methods, the responses to research questions 2 and 3 are based on surveys o f professional staff, faculty, and administrators — and, to a lesser extent, on the analysis o f N W I C documents. Surveys, participant-observation, and document review w i l l be discussed further i n the following section.  Surveys Structured surveys can be used to target specific topics and can provide valuable insight. A t the same time, there is a danger that bias or inaccuracy can be introduced into the process due to a number o f reasons: survey questions may be unclear or imprecise; respondents may remember inaccurately or provide dishonest responses; and the relationship between the researcher and the respondent may be poor. In this case, many staff, faculty, and administrators and the N W I C Institutional Review Board reviewed the survey questions and provided valuable input. In addition, I protected anonymity.  Participant-observation One advantage o f participant-observation as a means o f collecting information is that the observations o f real activities and behaviours take place in context. In addition, this method is unobtrusive and provides access and insight into people's struggles i n challenging situations (Stake, 1995). The disadvantages o f the participant-observation method include that it can be time-consuming and selective and the possibility that the presence o f an observer may influence or even change the course o f the events being observed. Furthermore, a bias may result due to the researcher's manipulating events ( Y i n , 2003, p. 86). Most o f these disadvantages were minimized i n this case study because I am an N W I C employee and my observations are a byproduct o f m y work.  143  Document review Document review involves reviewing relevant documents to the research i n order to make inferences and to verify basic information ( Y i n , 2003). Sometimes it provides information that is unanticipated (Stake, 1995). The advantages o f document review are that documents are permanent, secure, unobtrusive, and provide a wide coverage over time and o f events. The disadvantages include that they can be difficult to retrieve, may be incomplete and, therefore, biased, the author may be biased, or access might be denied. In this study, I reviewed and referred to N W I C ' s accreditation reports, strategic plan, and survey reports. Because each o f these methods (i.e., surveys, participant-observation, and document review) has advantages and disadvantages, construct validity is maximized when several are used together. Furthermore, when both the researcher's perspective and the participants' perspectives are considered, the research tends to be better informed.  Internal Validity Internal validity is the extent to which a condition can be determined to be caused by another condition (Stake, 1995). The researcher can address this concern during the data analysis stage by comparing results with expected outcomes for consistency (Stake, 1995), by building explanations, by addressing alternative explanations or interpretations, by being attentive to all the information, and by using logic models (such as tracing events over time in a flow chart) ( Y i n , 2003). In addition, triangulating data can minimize threats to internal validity. Internal validity is more o f a concern for explanatory or causal case studies.  144  External Validity External validity is the extent to which a study's conclusions or findings can be generalized to another population or situation (J. G a l l et al., 2005; M . G a l l et al., 2003). A s mentioned earlier, empirical data from case study research cannot be generalized to other cases; however, findings in a case study can be generalized to theory, and this theory can be used to examine other cases (Flyvbjerk, 2006; Y i n , 2003). Therefore, a theoretical proposition  can  orient the analysis ( Y i n , 2003, p. 111). Stake (1995) states that "valid modification o f generalization can occur i n case study" (p. 8) research; therefore, the researcher should "indicate how the findings might be extrapolated, how they could be interpreted i n various circumstances, and how they accommodate theoretical discourse" (p. 93). Following this logic, I can identify criteria from mainstream assessment research and determine which o f these criteria applies to assessment in a tribal college context. From these findings, theory can be generated that can be applied to other tribal colleges (Feagin et al., 1991). Marshall and Rossman (1999), L i n c o l n and Guba (1985), and Talbot (1995) discuss four ways to enhance the trustworthiness o f a study: credibility/validity