Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Learning outcomes approach in British Columbia’s colleges and university colleges Sunell, Susanne 2003

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


831-ubc_2003-854108.pdf [ 19.73MB ]
JSON: 831-1.0055601.json
JSON-LD: 831-1.0055601-ld.json
RDF/XML (Pretty): 831-1.0055601-rdf.xml
RDF/JSON: 831-1.0055601-rdf.json
Turtle: 831-1.0055601-turtle.txt
N-Triples: 831-1.0055601-rdf-ntriples.txt
Original Record: 831-1.0055601-source.json
Full Text

Full Text

LEARNING OUTCOMES APPROACH IN BRITISH COLUMBIA'S COLLEGES AND UNIVERSITY COLLEGES by SUSANNE SUNELL ' B.A., Queen's University 1972 Dip. D.H., University of Toronto 1974 M.A., University of British Columbia 1996 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION in THE FACULTY OF GRADUATE STUDIES (Department of Educational Studies) We accept this thesis as conforming to the required standard  THE UNIVERSITY OF BRITISH COLUMBIA March 2003 © Susanne Sunell 2003  In  presenting  this  degree at the  thesis  in  partial fulfilment  of  University of  British Columbia,  I agree  freely available for reference copying  of  department  this or  and study.  his  or  her  representatives.  permission.  Educa. -hvntJL  The University of British Columbia Vancouver, Canada  Date  DE-6 (2/88)  that the  may be It  publication of this thesis for financial gain shall not  Department of  requirements  I further agree  thesis for scholarly purposes by  the  Studies  for  an  advanced  Library shall make it  that permission for extensive granted  is  by the  understood be  allowed  head  that without  of  my  copying  or  my written  ABSTRACT This study investigated the suitability of adopting a learning outcomes approach as a strategy for educational reform in British Columbia's colleges and university colleges. It focused on the views of institutional and department administrators during the initial implementation phase through a questionnaire (n=313), interviews (n=58) and the analysis of provincial documents. Study participants had varied reactions to a learning outcomes approach ranging from strong support to overt resistance. Proponents viewed it as a philosophical shift from teaching to learning involving themes such as transparency, integrated curricula, holistic curricula, and a learner-centered focus. However, many viewed learning outcomes as being similar to their current approach. Opponents viewed the approach as being too simplistic, too limiting and unsupported by evidence. Its central position in the reform agenda was questioned. The barriers to its implementation included competing priorities, lack of resources, faculty workload, organizational culture, pedagogical issues, concerns about the vocationalization of postsecondary education and its perceived relationship to the provincial government's accountability movement. Approximately one third of respondents who had made changes identified them as valuable to their programs and courses. However, respondents from academic areas had less involvement, less interest in integration and perceived it as less valuable than respondents from applied areas. The value of the approach resonated at the theoretical level, but often disappeared in the practice context particularly at the course level. It was viewed as being particularly valuable in applied areas, but was most often described as a refinement. The learning outcomes approach was too abstract to provide a vision for reform. While there have been changes in specific courses and programs, the policy did not have a provincial impact from a pedagogical or accountability perspective. The term has been integrated into many organizational documents, but it is unclear if these changes translated into more relevant learning experiences or more valid assessment approaches. The discussions generated about best practices have been the greatest impact of the  ii  policy. It forced faculty members to challenge and defend their educational practices. This may be the ultimate legacy of the learning outcomes policy in British Columbia.  iii  TABLE OF CONTENTS  Abstract  ii  ;  List of Tables  vi  List of Figures  vii  Acronyms  viii  Acknowledgements  ix  CHAPTER ONE INTRODUCTION AND PURPOSE OF THE STUDY Background The Research Problem Research Questions Design and Methods General Outline  1 1 5 8 8 9  CHAPTER TWO GENERAL LITERATURE REVIEW Influences for Change Key Words in Educational Reform Learning Outcomes Approach The Policy Process Defined Summary  10 11 26 35 60 62  CHAPTER THREE  CANADIAN LITRERATURE AND BC POLICY CONTEXT Postsecondary Education in Canada Public Postsecondary Education in British Columbia Learning Outcomes Policy in British Columbia Summary  CHAPTER FOUR DESIGN AND METHODS Procedure Survey and Interview Instruments Participants and Non-participants Limitations and Delimitations Summary  iv  64 64 70 80 97 98 99 110 112 119 122  CHAPTER FIVE RESULTS Origins of the Learning Outcomes Policy Shaping the Learning Outcomes Concept Response to the Learning Outcomes Policy Outcomes of the Learning Outcomes Policy Summary  124 124 129 153 176 184  CHAPTER SIX DISCUSSION Defining Learning Outcomes in British Columbia The Implementation of a Learning Outcomes Approach Outcomes of a Learning Outcomes Approach Relationship of Learning Outcomes to the Strategic Plan Summary  187 188 195 205 212 216  CHAPTER SEVEN  SUMMARY, CONCLUSIONS AND RECOMMENDATIONS Summary of Study Conclusions Recommendations Concluding Comments  BIBLIOGRAPHY  Appendix A  Appendix B  218 218 220 224 227  229  Employability Skills Profile from the Conference Board of Canada  268  Comparison of Essential Abilities from International, National and Provincial Documents  269  Appendix C  The Study Sites  272  Appendix D  List of Events Surrounding This Study  280  Appendix E  Survey Instrument  292  Appendix F  Interview Guide  299  Appendix G  Organizational Target Sample and Respondent Data  302  Appendix H  Statistical Data  311  v  List of Tables  Table 1  How are Learning Outcomes Different from Behavioral Objectives / Competencies  40  Table 2  Terms Used to Refer to the 21 Century Skills  43  Table 3  Analysis of Participants in Provincial Dialogues  87  Table 4  Relationships Between Policy Texts in Charting a New Course  92  Table 5  Summary of Data Collection Methods  103  Table 6  Characteristics of Target Sample (R=709) and Survey Participants  st  (n=313)  113  Table 7  Interview Information from the Four Sites (n=36)  117  Table 8  Interviewees Based on Program Areas of Organizational and Department Administrators (n=33) Overall Interviews Conducted (n=58)  118 119  Table 9  vi  List of Figures  Figure 1  Context of Policy Making  61  Figure 2  Primary Elements Required in the Future Learning System  90  Figure 3  A Conceptual Framework for Post-Secondary Reform  96  Figure 4  External and Internal Influences on the Learning Outcomes Policy in British Columbia as Perceived by Study Participants  125  Figure 5  Learning Outcomes as Defined by Study Participants  141  Figure 6  Themes from Interview Data Related to Accountability  163  Figure 7 Figure 8  Study Participants' Views Related to the Message in Charting a New Course Focus of Policy Analysis of Learning Outcomes Study  167 187  Figure 9  Shaping of the Learning Outcomes Concept in British Columbia ....190  vii  /  Acronyms  BTAA  Budget Transparency and Accountability Act  C2T2  Centre for Curriculum, Transfer and Technology (also referred to as "the Centre" )  CCPD  Centre for Curriculum and Professional Development (reorganized into C2T2)  CSAC  College Standards and Accreditation Council (Ontario)  DPRC  Degree Program Review Committee  LON  Learning Outcomes Network  LOC  Learning Outcomes Coordinator  MoAE  Ministry of Advanced Education  MoAETT  Ministry of Advanced Education, Training and Technology  MoEST  Ministry of Education, Skills and Training (created in 1996)  MoSTL  Ministry of Skills, Training and Labour (1993 to 1996)  PLARC  Prior Learning Assessment and Recognition Coordinator  SCOEA  Standing Committee on Evaluation and Accountability  UNBC  University of Northern British Columbia  viii  Acknowledgements I would like to extend a special thanks to the following individuals for their insights and support through this journey.  •  To Tom Sork for his encouragement, analysis and guidance as the chair of my committee during this and my previous journey in graduate studies.  •  To Kjell Rubenson for his on-going nudges, frank comments, and consistent interest throughout this project.  •  To Judith Ottoson for her encouragement and support during the initial phase of my research.  •  To Lesley Andres for her detailed and comprehensive analysis of my work.  •  To Garnet Grosjean for his editorial skills, emotional support and pragmatic advice during the difficult times.  •  To the many participants in my study who shared their experiences and insights so freely. Their words of encouragement during the final stages of my writing were invaluable.  •  To the faculty and administration at Vancouver Community College for their support.  •  To the members of my cohort, the swampdogs, for showing me the trail and encouraging me along the way.  •  To my partner, Larry Sunell, for this patience, humour and unconditional love throughout this long journey.  Thank you all for sharing your strengths with me.  ix  CHAPTER ONE INTRODUCTION AND PURPOSE OF T H E STUDY  A learning outcomes initiative was introduced into the college, institute and agency system of British Columbia by the Ministry of Education, Skills and Training (BC MoEST) in 1996. The Centre for Curriculum, Training and Technology (C2T2), a nonprofit, independent society, was directed to promote a learning outcomes approach within this educational sector. Such an approach was described as important in furthering a learner-focused education system and in promoting the overall goal of educational reform. This study addresses the suitability of a learning outcomes approach as a strategy for educational reform in the BC College and university college sector. In this chapter, I address the key issues of accountability and performance measures in post-secondary education, and investigate their role in the emergence of the learning outcomes initiative. The study focuses on the period following the release of the Ministry's 1996 Strategic Plan for the College, Institute and Agency System, and examines the role of educators and administrators in the college and university college sector as they attempted to implement the policy. Then, I describe the research problem, outline the purpose of the study and present the research questions. Finally, I introduce the research design and methods of data collection, followed by the structure of the thesis.  Background Education has long been seen as a vehicle to promote the development of human capital (Schultze, 1961; Marginson, 1997a). The forces of globalization and the evolution of a knowledge society increased the need and demand for higher education (Kenny-Wallace, 1988). However, higher education did not fulfill the expectations for sustained economic growth as many had expected (Levin & Kelly, 1997; Avis, 2000). As national economies entered periods of decline, questions arose about the accountability of public sectors in general and the educational sector in particular. The influences for change predominantly rosefromthe following: the state of national economies, the  1  demand for more opportunities for higher education, and concerns about the credibility of higher education (Coate, 1995). While evaluation had always been part of higher education, the emergence of the evaluative state not only entailed a shift to outcomes, it also altered the focus of evaluation from system maintenance to evaluation for strategic change (Neave, 1988). The pressure for increased accountability was evident in many western nations and developing nations as well (Bruneau & Savage, 2002). The pressure for accountability was often couched in corporate language with a focus on efficiencies, effectiveness and cost analysis. The previous forms of evaluation that often included reputational models and self-assessment were questioned. Government ministries and funding organizations came under increased pressure to justify their decisions, to develop criteria for measuring the value accrued from resource allocations (Cave, Hanney, & Kogan, 1991). In the United Kingdom the concept of accountability was operationalized through the development of performance indictors to promote greater quality and efficiency in higher education. Australia and New Zealand pursued a similar route (Marginson, 1997a; Bruneau & Savage, 2002). Credentialing functions became a high priority in many countries, but particularly so in the United Kingdom, New Zealand, and Australia (Department of Education, 1993; Curtain & Hayton, 1993; Fitzsimons, 1999; Yorke, 2000). In the United States the conversations focused on similar themes but they were operationalized through the notion of assessment that was furthered by the accreditation system (Derlin, Solis, Aragon-Campos, & Nidella, 1986). The assessment movement was driven by the recognition that the needs of our society and our learners were changing with the shiftfroman industrial age to an information age. A knowledge-based economy required workers who were more highly skilled and adaptable (Wingspread Group on Higher Education, 1993). Pressure for changes came from external sources and from within educational organizations (Mentkowski, 1998). The discussions surrounding educational reform focused on the quality of programs, particularly undergraduate education, and the quality of teaching (Wingspread Group on Higher Education, 1993; Gardiner, 1994).  2  Similar conversations about accountability and quality also occurred in the Canadian context (Smith, 1991; Benjamin, McGovern, & Bourgeault, 1993; Dennison, 1995). In British Columbia the changing needs of the labour sector and their implications for skill development were highlighted in Training for What? (British Columbia Labour Force Development Board, 1995). This report focused on the "skills gap," "relevance gap" and "accountability gap" between the economic and educational sectors. The employability skills profile developed by the Conference Board of Canada (1992) was suggested as a means to address the need for a more highly skilled workforce to sustain the economy of British Columbia. The proposed policy directions revolved around student outcomes, employer outcomes and outcome-based accountability frameworks. The learning outcomes initiative in British Columbia emerged from these conversations as one piece of a larger public policy puzzle. In the texts of the BC strategic plan, Charting a New Course: A Strategic Plan for the Future of British Columbia's College, Institute and Agency System (BC Ministry of Education, Skills and 1  Training , 1996) learning outcomes appeared to be an important piece. In the literature from C2T2 it was presented as the key element in the design of an educational system J  that would promote a flexible, seamless, and cost-effective educational system. Learning outcomes were deemed to be central to educational reform in British Columbia and C2T2 allocated funding to promote their implementation within higher education in BC. The concept of learning outcomes evolved from two areas, although the two conversations often merge in practice. Firstly, the concept was linked to political discussions about accountability and the assessment of the outcomes of learning. Secondly, it was also based in concerns about pedagogy. From this perspective it was described as a strategy to promote coherence (Mentkowski, 1998), clarity and transparency (Avis, 2000). This aspect originated in part with the literature on Outcomes Based Education (OBE) that emerged in response to the mandate of secondary education to create "good citizens" and "good employees" (Spady, 1994; McGhan, 1994). The OBE movement grew out of concerns that American high school graduates did not posses  This document will be referred to as Charting a New Course, the name commonly used in BC. This Ministry will be abbreviated to MoEST in future citations. Through this strategic plan this organization was directed to promote a learning outcomes approach within this educational sector. 1  2  3  3  the skills and knowledge to integrate into economic and community life. Graduate learning outcomes were described as "high-quality, culminating demonstrations of significant learning in context" (Spady, 1994, p. 18). Learning outcomes were defined in broad, general terms so as to reflect cumulative learning upon graduation from an educational program. This learning was intended to be reflective of "real world" life. It seems logical to define what you expect students to learn, and then to design the instructional activities and the evaluation structure around those defined learner outcomes. Tyler (1949) addressed these ideas many years ago; they are hardly new or innovative. Educators have been discussing outcomes for many years in relationship to course goals, instructional objectives, behavioural objectives and competencies. We have a plethora of terms and definitions that are all basically related to the purpose or aims of education. Allan (1996) notes that educators moved from the general education objectives described by Tyler to more discrete objectives such as "instructional objectives" and "behavioural objectives." The pendulum is now swinging back to general statements. The current focus on "learning outcomes" reflects a shift along a continuum, from specific to general outcome statements. A shift towards more general outcome statements may have been warranted, particularly in the applied programs, but such a shift did not appear to warrant its central location within the reform vision for BC higher education. It was challenging to determine what was meant by a learning outcomes approach in British Columbia. At the C2T2 retreat on Bowen Island, (January 30, 1998) Ruth Stiehl from Oregon State University spoke about the unique, holistic, integrated characteristics of a learning outcomes approach. However, there was no consensus achieved at the meeting regarding the notion of learning outcomes. Some participants viewed it as a specific model, others as a philosophical approach to learning. There appeared to be considerable confusion about the notion of a learning outcomes approach. This signaled the need for further investigation. We appeared to be on a journey to promote a learning outcomes approach in college, institute and agency sector in BC, but without a clear understanding of what it was. The proposed destination was unclear. The ensuing dialogues focused on the "how" aspects of learning outcomes. For example, proponents discussed how to develop learning outcomes, and how to evaluate them. The "why" question was only superficially addressed in C2T2's documents and  4  articles of the day. Rhetoric abounded regarding the value and relevance of the learning outcomes approach. It was described by some (Shipley, 1994a, 199b; Battersby, 1997; Bauslaugh, 1997a, 1997b) as dynamic and innovative. In a letter to Education Councils, Battersby and Malnarich presented the initiative in the following way: "For a coherent and effective post-secondary system, learning outcomes must provide the concept that links funding initiatives, technological innovation, flexible assessment and pedagogic change" (personal communication, October 30, 1997). There appeared to be many assumptions behind the decision to direct resources towards a learning outcomes approach. These assumptions needed to be analyzed and questioned more thoroughly. It was not clear how a learning outcomes approach would lead to the changes described in C2T2's literature.  The Research Problem In 1994 I was involved in a national workshop held in Winnipeg that focused on the development of national education standards for dental hygienists. Two federal government representatives attended our meeting as observers; they were interested in our discussions from a labour mobility perspective. During the course of the discussions, a representative from Ontario suggested that we shiftfromour competency framework to a learning outcomes approach. That was my introduction to the term 'learning outcomes'. I had been involved in post-secondary education for over twenty-two years as a faculty member, department administrator, consultant, and as the Health Sciences representative to Education Council during its formative years. I had also actively participated in provincial and national policy decisions pertaining to dental and dental hygiene education, examination, regulation, and accreditation. Given this background, I was intrigued by the notion of a different approach to curriculum and assessment. At the Winnipeg meeting we struggled to understand the concept. It seemed so similar to what we already had. My interest was further peaked when I was invited to participate in the validation process for learning outcomes for diploma dental hygiene programs through the Ontario College Standards and Accreditation Council. The documentation suggested that these outcome statements represented the results of  5  extensive investigations but they appeared simplistic and self-evident to me. I could not grasp their innovative characteristic. Subsequently I received contracts for two other projects that integrated learning outcomes. The first involved the development of the licensure mechanism for a new BC Residential Care License for dental hygienists; the other involved the articulation of outcomes for BC diploma dental hygiene programs. In both cases the learning outcomes approach was used as an instrumental strategy to facilitate communication among diverse interest groups in the first case, and to gain funding for a provincial dialogue in the second case. While the approach proved useful for instrumental reasons, I was still perplexed by its relationship to educational reform. In October 1997, C2T2 sent an invitation to BC public post-secondary education institutions requesting their participation in a Ministry funded learning outcomes program, a Learning Outcomes Network that was to promote a learning outcomes approach in order to achieve the following goal from Charting a New Course: "To provide British Columbians with post-secondary education and training to improve the quality of life and citizenship experienced in the province and to enhance current and future job opportunities" (MoEST, 1996, p. 31). The Learning Outcomes Network was seen as vehicle for "implementing a systems wide exploration of learning outcomes" with one of the network's projects being "the development and articulation of general education outcomes" (personal communication, Battersby & Malnarich, October, 30, 1997). The invitation was accompanied by an offer of partial funding to establish a coordinator position within each participating organization. The funding provided onequarter release time for a Learning Outcomes Coordinator for a six-month period with the possibility of future funding in the subsequent fiscal year. The Learning Outcomes Coordinators were to be supported in several ways including a variety of support services, conferences, workshops, on-line discussion group and a web site. This provincial network of coordinators was viewed as a group to facilitate the collaborative development of a learning outcomes approach, which would be the central reform element in the creation of a seamless, flexible, and learner-centered post-secondary education system (Bauslaugh & Hansen, 1996).  6  The administrators and educators at my college had mixed views about the 4  invitation. Some were intrigued and some were skeptical; others were simply offended. Some saw the initiative as a strategic policy shift; others perceived it as an operational strategy for flexible assessment or the latest trend in a long line of ministry generated panaceas. Many were confused, perplexed and bewildered about the initiative; they wondered why funds were being directed to this initiative during times of fiscal restraint. They also questioned the need for faculty to devote time to this issue given the many other competing priorities they faced. The rationale for the initiative and its relationship to educational reform was not readily evident to many within the college and university college sector. The purpose of this study is to investigate the issues surrounding the learning outcomes initiative in BC colleges and university colleges. In particular, the study is designed to increase understanding of the policy during the implementation phase, as educators and administrators in the college and university college sector attempted to achieve the objectives of the strategic plan. As a practitioner affected by this policy I was interested in gaining a better understanding of the learning outcomes approach. I felt it was important to go beyond the rhetoric, and determine how educators and administrators understood the policy and how they approached it in their practice environments. The results of this study will allow faculty members in the college and university college sector to better understand the learning outcomes initiative and the nature of provincial policies in British Columbia. Faculty members are so involved in the daily activities associated with teaching and learning that it can be challenging for them to gain a perspective on the larger provincial context. They are often isolated within their program or discipline areas. The results of this study may help them gain a better understanding of their working world and its relationship with other areas both within their own organizations and within their provincial communities. The results will also provide administrators with insights into key issues in the development and implementation of policy directions for BC colleges and university colleges. The perspectives and understandings arising from this study may also be helpful to future  1 was the Health Sciences representative to the Vancouver Community College Education Council when this policy was introduced.  4  7  researchers interested in analyzing the ongoing metamorphosis of post-secondary education in British Columbia.  Research Questions This study investigates the suitability of adopting a learning outcomes approach as a strategy for educational reform in the BC colleges and university college sector. I wished to explore the learning outcomes initiative from its apparent inception in a New Course  Charting  to its implementation in BC colleges and university colleges. The study  focuses on the 9 to 12 month period after the funding of the Learning Outcomes Coordinators in 1997. The study is guided by three broad questions: (1) How was the concept of learning outcomes being defined in the colleges and university colleges? (2) How did the people in this sector view this policy direction? (3) Was the learning outcomes initiative helpful in promoting the vision described in the strategic plan of the BC Ministry of Education, Skills and Training? A critical analysis of this policy was warranted given that public funds were directed towards its promotion at a time of fiscal restraint in higher education.  Design and Methods The study participants were drawn from the organizations that opted to participate in the Learning Outcomes Network. Of the 21 educational institutions participating in this network, 20 represented the college , institute and agency sector. From this group of 5  20 organizations, I selected the 16 colleges and university colleges as my target group since they shared a common mandate. The study consisted of several phases. The initial phase included a survey of organizational and department administrators, Learning Outcomes Coordinators (LOCs) and Prior Learning Assessment and Recognition Coordinators (PLARCs). The survey data were augmented by interviews with the survey respondents as well as students and  The term "college" refers to community colleges and university-colleges in the language of the strategic plan. 5  8  other key players from organizations linked to postsecondary education. Organizational documents, minutes of meetings and email discussions on the Learning Outcomes Network were also analyzed to gain additional insights into the discussions within the colleges and university colleges about curricula and educational reform issues.  General Outline My goal in conducting this study and communicating the results was to engage practitioners in a dialogue about a learning outcomes approach specifically and the broader educational change process in general. This chapter has provided an introduction to the study. Chapters II and III provide background for understanding the issues from a general and provincial perspective. Chapter IV presents the research design, and methods of data collection and analysis. This is followed by the presentation of the interview and survey data in Chapters V. Chapter VI includes the discussion and Chapter VII presents the conclusions and recommendations arising from the study. My aim was to link theory with practice to facilitate a broader discussion of learning outcomes in the BC college and university college sector.  9  CHAPTER TWO GENERAL LITERATURE REVIEW The learning outcomes initiative in British Columbia was designed to promote a seamless, student-centered educational system (Bauslaugh & Hansen, 1996). To understand this initiative, we must also understand the influences on its development. At the heart of these influences were discussions about the relationship between education, and the economy and the role of government in the educational system (Rae, 1988). Economic renewal and intellectual development were deemed to be inextricably linked (Kenny-Wallace, 1988). Therefore, managing the forces of globalization and adapting to the needs of a knowledgebased society became key challenges in maintaining a competitive edge in international markets (Kenny-Wallace, 1988; Watts, 1988). Post-secondary education was perceived as an essential element in furthering this competitive edge and thus promoting economic renewal and prosperity (OECD, 1997a; Marginson, 1997a; Ainley, 1998). Debates about the need for educational change revolved around the development of human capital and social capital with education seen as a vehicle for economic prosperity, social mobility and equity. Some perceived a tension between education for economic development and education for social justice (Rae, 1988; Soucek, 1993); others suggested that these two could be complementary (Paquet, 1988; Morse, 1988). However, both influences affected the demand for education. The state of the economy and the increased demand for education focused attention on the productivity and accountability of higher education. Governments demanded that educators be more accountable for the use of public funds (Cave et al., 1991). This led to a search for indicators of performance that would allow governments to evaluate and measure the outcomes of learning. I begin this chapter by looking at the influences that surround the call for reform in higher education and the key concepts commonly discussed in relationship to such reform. I then analyze the literature on learning outcomes in general, and review the relationship between learning outcomes and educational reform. The chapter concludes with a discussion of policy, the policy process and the framework that will support my analysis of the learning outcomes initiative. This chapter complements the Canadian and BC literature that will be the focus of Chapter Three; the two literature chapters are designed to provide a context for understanding my study.  10  Influences for Change Changes in advanced education are largely driven by economic factors including the forces of globalization and human capital theory (Marginson, 1997a). These changes were strongly influenced by advances in technology, which enabled the shift from an industrial to a knowledge economy (Zemsky, 1996; Puyear, 1997). In this section I describe the developments that helped shaped the demand for increased accountability and quality in higher education. The discussion will focus on the development of a knowledge economy and society, the forces of globalization and how these forces effect change in the governance of education.  The Knowledge Economy and Society We have experienced a rapid growth in the use of information technology and other forms of communication in recent years (Rubenson & Schuetze, 2000). This facilitated the growth of a global market exchange and has transformed many aspects of our world (Marginson, 1997b). The key elements of this shift from an industrial age to an information age are described by Aronowitz and De Fazio (1997). Scientific and technological innovation is, for the most part, no longer episodic. Not only has abstract knowledge come to the center of the world's political economy, but there is also a tendency to produce and trade in symbolic significations rather than concrete products. Today, knowledge rather than traditional skill is the main productive force, (p. 194) As a commodity that can move freely across national boundaries, knowledge has become an important source of power and wealth (Brown & Lauder, 1997). Lipsey (2000) contends that we need to fully recognize the non-rivalrous characteristic of knowledge as this characteristic makes it unlike other types of commodities that have supported economic growth; its consumption does not preclude the consumption by others as do may products such as apples and chocolate for example. The educational needs of individuals in the knowledge age are different from those of an industrial age (Drake, 1997; Candy, 2000). Science and technology have changed the forms of work and employment (Rubenson & Schuetze, 2000). Technology is changing the way we work, live and learn (Zemsky, 1996; Puyear, 1997). The rapid development and diffusion of information and communication technologies have influenced the growth of post-secondary education, resulting in a shift from elite to mass higher education (Alexander, 2000). The time between compulsory education and entry into the labour market has  11  expanded (Williams, 1999) and learning is no longer seen solely as the function of institutions of higher education; it permeates our lives. Usher, Bryant and Johnston (1997) argue that this shift to a learning society is one of the biggest challenges facing our educational organizations today. Forces of Globalization Through television and other media we are reminded of the ubiquitous effects of globalization. Marginson (1997b) argues that globalization is not just about the internationalization of goods, services, money, people and ideas, but encompasses a wide range of relationships and inter-connections between states involving both bi-lateral and multi-lateral connections. Globalization is about world systems which have a life of their own that is distinct from local and national life, even while these world systems tend to determine the local and national. ... Globalization is complex and multiple, embracing practices which are conventionally described as 'economic', 'political', 'sociological', 'cultural', 'linguistic, 'semiotic' and so on. (p. 20) This definition highlights the complex nature of the phenomenon. Globalization is not necessarily a homogenizing force; it also provides opportunities for diverse cultures to coexist (Henry, Lingard, Rivi, & Taylor, 1999). It promotes increased Anglo-Americanization while also promoting diversity through multilingualism and multiculturalism (Marginson, 1997b). However, globalization with its adoption of market liberalism has also widened the gap between socio-economic groups within nations (Ball, 1994). Globalization has increased the power of capital in public policy (Howlett, & Ramesh, 1995). Financially poorer states are particularly susceptible to the influences of international capital and its movement across national borders. Marginson (1997b) argues that there is a "lack of fit" between global markets and global culture, and national politics. There is often a disparity between the political and ideological stances taken by governments. Governments oscillate between blaming external factors and refusing to acknowledge them. It is often easier for governments to support deregulation as national policies may be at odds with the global market economy. By freeing up the market forces they hope to gain an advantage in the global economy. However, this presumed advantage comes at a cost for some members of society; it widens the gap between those with power, resources and privilege (Ball, 1994). Market liberalism systematically  12  disadvantages marginalized groups and communities. It has influenced the erosion of welfare services and increased the income disparity among segments of society (Ainley, 1998). The effects of globalization are mediated through national patterns and structures which Dale (1999) labels societal and cultural effects. He points out that nations adopt one of two common approaches to globalization. One is a competition form where the nation attempts to increase its competitive advantage, and the other a governance without government form as described by Rosenau (1992). The latter form refers to the establishment of a framework of international organizations that are involved in performing functions for keeping systems viable. While the work of these organizations is diverse, consensus has been achieved for a common ideological approach to issues. These include, but are not limited to, the concepts of financial liberalization, trade liberalization and deregulation (Dale, 1999). Both the competitive and the governance approach have implications for educational policy. Dale notes that the competitive approach has brought economic policy and initiatives to the foreground while discussions about equity have become marginalized or encompassed in other agendas (Henry et al., 1999). Politicians tend to ignore the influence of globalization in some circumstances, and then blame the failure of their policies on it in others (Marginson, 1997b). We appear to have an uneasy relationship with globalization at best. Dudley (1998) in fact questions the validity of the claim that national economies are being subsumed by the global economy and its attendant international markets. She argues that "globalization is a discursively constructed master discourse of uncontrollable global market forces that valorizes the economic rationality of neo-classical economics and the minimalist politics of neo-liberalism" (p. 30). Howlett and Ramesh (1995) also suggest that the influences of globalization are exaggerated and are often more prominent at the macro than the micro area. However, they also contend that the degree of influence is dependent on nations' economic strength and the public policy of poor countries is, therefore, more influenced by international money markets. While the influences of globalization are ubiquitous, Marginson (1997b) identifies other factors that are also at play. The first of these is the trend to "civic universality," the trend to support the involvement of individuals in citizenship, learning, labour markets and consumption. "The key figure in late modem systems of government is the self-regulating, choice-making, self-reliant individual" (p. 25). However, Furlong and Cartmel (1997) suggest that risks for marginalization still continue to be distributed on the basis of social class (as expressed through educational performance) and gender. A second factor identified by  Marginson is the increased trend for governments to promote changefroma distance. This trend is a reflection of a market liberal orientation but is also supported in social democratic regimes. Karmel (1996) discusses these issuesfromthe perspective of a change in the economic paradigm over the last 20 years. The role of the public sector in economic development has been de-emphasized and greater weight is being placed on individualism, entrepreneurship and market forces. "We have moved from a protected, inward looking, much regulated economy towards a much more competitive one with a global outlook" (p. 25). These influences in combination with the influences of globalization have forced governments to change the way they operate. The work of the nation-state today has become embedded in a web of international relationships; the state is but one player among many often overlapping and competing organizations with an influence and an interest in economic, social and political spheres (Dale, 1999). Economic relationships have changed but the state must still provide the legal and social parameters for the operation of the national economic market (Marginson, 1997b). The parameters have changed, but the nation-state still has an important role to play in the social and economic support of its citizens (Ainley, 1998; Henry et al., 1999), and in the relationship between education and the economy (Rubenson, 1987).  Governance  of  Education  Globalization has transformed the relationship between the state and education leading to the rise of the evaluative state (Neave, 1988). This involves a shift from routine evaluation to strategic evaluation, and a reorientationfromthe more classic input, process, and output model to a model that emphasizes the evaluation of outcomes. Neave suggests that the emergence of the evaluative state brings into question the dichotomy between centralization and decentralization. "In systems based on decentralisation, the Evaluative State appears as a step towards greater central control and, in those based on a higher degree of centralism, it is perceived as giving rise to greater flexibility and hence greater decentralization" (p. 11). This suggests that by shifting the focus to measuring outcomes, institutions may in fact acquire a greater degree offreedom,but clear lines of accountability are required to ensure a focus on national goals. Harden, Lewis and Graham (1992) propose the concept of a contracting state. From this perspective states are viewed as contracting out their services to a combination of private,  14  semi-private and public providers. However, the state still holds the power to contract and thus controls the services provided. The state's financial control is increased, but the arm's length relationship results in less control over details. Ainley (1998) suggests that such contracting relationships in education are highly unstable due to the complexity of organizations in higher education and national goals involved in this policy area. Karlsen (2000) raises the distinction between decentralization as delegation versus devolution, where devolution implies a shift in power and authority. In Karlsen's distinction of delegation the central authority still defines the priorities; only the tasks and responsibilities are shifted to the periphery. Ball (1994) raises a similar issue. He contends that the discourse on self-management is legitimized through a discourse on autonomy. However, this autonomy may be the autonomy of managers and may be a constraint on faculty members. Kells (1992) supports this position and suggests that the power in an educational system may have shifted from government ministries to institutions, but the power may have only shifted to the organizational leaders and not the faculty members. Policy makers may suggest the liberating effects of this evaluative approach, but educators may only experience increased accountability and reporting mechanisms. While the current application of decentralization may not reflect a shift in power, there may still be opportunities for increased autonomy as it is challenging to exert total central control. Karlsen suggests the term "decentralized centralism" to more accurately convey the reality of this type of government approach. Dale (1997) notes that the most common response of states to the difficulties encountered in education has been to withdraw. Public organizations are expected to raise funds through corporate involvement and through consumers of education. The discussions are couched in corporate language; educators become producers, and educational administrators become managers and entrepreneurs (Marginson, 1997b; Avis, 2000). Government texts focus on the themes of privatization and decentralization (Karlsen, 2000). Dale (1997) argues that this withdrawal is a matter of expediency to promote reduced public funding, and to improve the efficiency and responsiveness of education. States have basically rejected their commitment to guarantee education as a public good. Providing citizens with opportunities to participate in advanced education is no longer seen as the responsibility of governments. Instead governments have attempted to insert a competitive ethic as the main driving force for the development of higher education (Neave, 1988). Education has become a market steered by government interventions (Marginson, 1997a).  15  Dale attributes this marketization of education to a shift towards a neo-liberal ideology, and a response by governments to their perceived capacity or incapacity to effect change in the present social and economic context. Supporting deregulation is often an easy route for governments to avoid having national policies in conflict with the global market economy. These various discussions reflect the argument by Karlsen (2000) related to decentralization as delegation or devolution. Governments suggest that an evaluative approach allows for increased opportunities and autonomy by higher education, but this approach at the same time increases the accountability and reporting function that are then often linked with funding. Development of Human Capital The 1960s and 1970s saw a trend toward the democratization of education (Marginson, 1997b). National education became a vehicle to develop a more just and fair society while also supporting the development of individuals' knowledge and abilities for economic gain (Fisher, Rubenson, & Schuetze, 1994; Brown & Lauder, 1997; Levin & Kelly, 1997) . This is often referred to as the human capital approach. Such an approach makes a distinction between education as consumption and investment (Schultz, 1961). As a longterm investment, education is perceived as an instrumental tool for economic well-being (Woodhall, 1997). Raising the quality and productivity of human capital is regarded as a competitive advantage (Brown & Lauder, 1997; Ainley, 1998). While Schultz acknowledged the challenges with this approach, it has been used for many years to categorize education based on its purpose or outcome. Applied education is often couched in investment terms while liberal education is seen as consumption - the benefits of a good general education despite the fact that it can also be viewed in investment terms (Nordhaug, 1987). The human capital approach supports increased investments in post-secondary education, and applied education in particular. Education policy became linked with economic policy (Ainley, 1998; Dudley, 1998; Karmel, 1996). Providing access to post-secondary education was high on the agendas of many nations. In an attempt to sustain and support the growth of their economies during times of increasing global competition, many nations began to provide increased opportunities for access to post-secondary education (OECD, 1998b). But the increased investments in education did not always produce the anticipated economic growth (Ainley, 1998) . Participation rates in education increased, but graduates were confronted by decreased  16  opportunities in middle range occupations (Dwyer & Wyn, 1999). Economies stagnated and unemployment figures rose, even when economic indicators suggested economic recovery (Aronowitz & De Fazio, 1997). Amid this economic decline came increasing demands from . the private sector, governments and educators for change in higher education (Paquet, 1998; Cutt&Dobell, 1988). In the 1980s and 1990s the project for socio-economic equalization began to wane amid pressures from liberal fiscal policy and the influences of globalization, and it was replaced by the notion of regulated educational systems to support merit-based selection of future leaders. Education had long been viewed as the factor that would break the link between social origin and social destination. However, the belief that government interventions could neutralize the social aspects of educational competition was questioned; concerns were expressed that the trend towards meritocracy would exclude currently • marginalized groups (Marginson, 1997b; Dwyer & Wyn, 1998). Henry et al. (1999) emphasize the need to put social capital back on educational agendas. The second wave of the human capital approach emerged in the 1980s with a focus on lifelong learning. Ainley (1998) suggests that it has become almost obligatory to include the notion of a learning society in discussions about economic prosperity. In this form of human capital, building the capacities of individuals for learning and self-reliance is seen as promoting their ability to adapt to changing market and labour trends (Watts, 1988; Marginson, 1997a). Education evolved from a strict career preparation focus for the privileged and the young, to a lifelong learning orientation for all segments of society. This focus on lifelong learning expanded the boundaries of the discussion about the benefits of education. Dyke (2000) suggests that lifelong learning has become a mantra for policy makers. He questions the rhetoric of empowerment and suggests that it is narrowly defined in policy to the point that it should more aptly be called lifelong training; Jarvis (2000) uses the term worklife learning. Adult learning is increasingly linked with training, and in many cases is used for the acquisition of specific abilities deemed important to economic development (Usher et al., 1997). Rather than creating lifelong learners, the policy agenda in education may be to create a lifelong learning market (Jarvis, 2000). Regardless of the language used, the concepts of human capital development remains central to public policy (Dudley, 1998). As Skolnik (2000) notes, instrumental economic objectives have been present for many years; post-secondary education has navigated  17  "between developing people and developing workers, between advancing knowledge and advancing industry" (p. 3). Many countries are investing more in education, particularly postsecondary education (OECD, 2000) even though the relationship between educational expenditures and outcomes was not found to be linear in the OECD analysis (1997a). This non-linear relationship raised questions of cost-effectiveness. Fisher and Rubenson (2000) detect a shift in the OECD conversations about human capital, from a macro to a micro perspective. As the skepticism about post-secondary education persisted, it led to conversations about the accountability and quality of higher education.  Calls for Accountability  and Quality  In times of increased fiscal pressures, governments continue to look toward educational reform as a solution to economic problems. However, governments also want to know what they are getting for their investment in higher education (Cutt & Dobell, 1988; Brennan, 1997; Jones, 1997; Skolnik, 1997). The return on educational investment for individuals is more obvious; the returns for societies are more elusive (OECD, 1997a). In their American study Haveman and Wolfe (1994) found that parental levels of educational attainment was associated with children's increased success and attainment. The probability for graduation from high school increases with increased parental schooling. This was more evident for children who had experienced poverty and for those who had lived in "bad neighborhoods." Their research supports the value of education for societies as well. With the increased needs and demands for education, governments and educators call for increased accountability within higher education. Many also call for change in the way higher education is delivered (Shugars, O'Neil, & Bader, 1991; Smith, 1991; Wingspread Group on Higher Education, 1993; Gardiner, 1994; BC MoEST, 1996; Bauslaugh, 1997). These arguments focus on certain themes: meeting learners' needs, attending to student learning, realigning structural elements, increasing the quantity and quality of educational research, and addressing the organization and relevance of curriculum. However, others question the arguments presented by the advocates of change (Cutt & Dobell, 1988; Gingras, Masse, & Roy, 2000; Birnbaum & Shushok, 2001). These diverse perspectives are summarized in the following section beginning with the arguments for change, and then followed by a discussion of the contested issues.  18  Meeting Learners' Needs Meeting the needs of learners is one of the central issues raised in discussions about educational change (Wingspread Group on Higher Education, 1993; Barr & Tagg, 1995). The characteristics of learners in postsecondary education have changed with the shift towards universal access in higher education. Learners today have more diverse backgrounds; many come to post-secondary education with years of experience in the workforce. While these learners are possibly more representative of a pluralistic society, they appear less prepared for higher education when compared to their more affluent peers; they bring different needs to educational institutions than previous students (Schroeder, 1993; Gardiner, 1994; Dunne, Bennett, & Carre, 1997; Kuh, 2001). These diverse learners are also more discriminating in the selection of educational opportunities; they raise more questions related to flexibility and access, (Rowley, Lujan, & Doence, 1998; Bridges, 2000) and thereby influence the argument for change in the current system. Focusing on Learning The need to focus on learning is a key element in discussions about educational reform (Lazerson, Wagener, & Shumanis, 2000). In Canada, the Smith report (1991) criticized universities for directing their resources to issues other than learners' needs. Smith argued that there were few innovations directed to teaching and that the pedagogical training of faculty members was not a priority. He concluded that teaching is seriously undervalued in universities. "Generally, the opinion in the university community seems to be that research technique takes years to learn but teaching simply comes naturally" (p. 60). While the focus of the report was directed to university education, the points regarding faculty members' knowledge and approaches to learning are equally applicable to the college and university college sector. Skolnik (2000) contends that the "sage on a stage" approach still represents our most frequent approach to intentional educational experiences. Nelson (1999) contends that American college faculty members have little understanding of the theory and practice of pedagogy, and have made minimal effort to remedy this situation. Dunne et al. (1997) conclude that the educators in their British study (n = 32) were not familiar with theories of learning. From a British Columbia perspective Gallagher (1995) points out that in the BC college and university college sector, "excellence in teaching has not been given the attention and support that was originally anticipated" (p.  19  260). A similar view is expressed by Grubb (1999) who notes the lack of institutional support for teaching within American community colleges. Grubb (1999) argues that the claims of community colleges as teaching institutions are more rhetoric than reality. His research suggests that college administrators know little about what happens in the classroom and much of our knowledge about teaching in community colleges is based on learner satisfaction surveys. Based on extensive observations and interviews in colleges, his research group concluded that the quality of teaching was dependent on the individual instructor. "The quality of teaching is individual and idiosyncratic, rather than the institutional responsibility of a teaching college" (p. 137). While there were examples of innovation, the instruction was often assessed as mediocre. Gardiner (1994) expresses concerns about teaching and assessment in higher education, calling for a clearer definition of intended outcomes with more focus on the interaction of curricula with student learning. This position is supported by others (Cross, 1997a, 1997b, 1998; O'Banion, 1997, 1999; Wilson, Miles, Backer, & Schoenberger 2000) indicating a need for better information about students' developmental needs and their achievements as they progress through the curricula. Realigning Structural Elements Barr and Tagg (1995) call for a shift in focus from teaching to learning. Although their argument is weakened by the use of a dichotomous framework for the relationship between teaching and learning, they raise valid points. Many of our current organizational structures and processes focus on educators' and organizational needs, rather than learners' needs. Courses and schedules are organized around specific disciplines and associated faculty needs. Access is often restricted by on-site delivery methods commonly implemented between the hours of eight to five. Learners are often forced to maneuver around various policies and procedures to have previous learning recognized in order to gain access to relevant courses. Skolnik (2000) supports the claim that many of our postsecondary organizations in Canada are designed for the people who administer and work in them. Grubb (1999) argues that our organizational structures are places where those who have been traditionally served by higher education do well. The ones who need more support are often blamed for their inability to cope within our structures; the responsibility is shifted from our educational organizations to the learners.  20  Cross (1999b) argues that educational structures are grounded in assembly line procedures, but our future lies in our ability to individualize educational experiences. Reform initiatives have been piecemeal and inadequate; therefore delivery tends to be fragmented rather than integrated and holistic. According to Gardiner (1994) important organizational changes can only come about through changes in the culture of post-secondary organizations. These perspectives about organizational influences and focus suggest the need to realign the structural elements of educational organizations to deliver learning that is appropriate for today's diverse groups of learners. . Educational Research Researchers suggest that change is challenging in post-secondary education when educational research receives little attention in the general university culture and context. For example, Smith (1991) is critical of the quality and paucity of research on the topic of higher education in Canada; Gallagher (1995) points to the lack of evidence about the quality of teaching in BC colleges and university colleges from a quantitative perspective. Papadopoulos (1998) and Kuh (2001) also stress the need for more educational research, and Gardiner (1994) argues that we tend to collect information about the intentions of curriculum in higher education but not the outcomes. Based on a study of British educators, Dunne et al. (1997) contend that educators know little about how students learn, although there is a sense that people learn in different ways. Donaldson (1999) argues that we base our teaching on research conducted on younger adults in transition from public school and that we have little understanding of variables that affect adult learning. This perceived lack of quality research may be related to the status of teaching and learning within organizations of higher education. Smith (1991) suggests that university faculty members tend to get more recognition for research in disciplines other than education. Wright (2000) expresses the same concern about assessment. "It has to be recognized as a legitimate contribution to scholarship and become a normal part of promotion and tenure consideration" (p. 56). The paucity of research is exacerbated within colleges, as research is not commonly a mandate. Dennison (1992) contends that the heavy student contact time in colleges presents a challenge for scholarly activity. He also suggests that the college environment breeds "intellectual fatigue" as faculty members teach the same introductory courses year after year, and often do not have the stimulation provided by association with colleagues teaching in  21  more advanced courses and graduate programs. Grubb (1999) notes that many colleges have offices of institutional research, but these offices are largely devoted to the generation of compliance reports and public relations efforts. When combined with the lack of institutional support for teaching; college faculty members are often left to fend for themselves in isolation.  Curriculum in Higher Education The organization and relevance of curriculum in higher education is also called into question (Wingspread Group on Higher Education, 1993; Dunne et al., 1997; Whitston, 1998; Bridges, 2000). The general education component is criticized for its lack of coherence and structure. "To express its perceived incoherent, hodgepodge character, this distribution system [certification based on credit accumulation] has been variously and irrelevantly dubbed a supermarket, cafeteria, grab bag, or green-stamp endeavor" (Gardiner, 1994, p. 34). Students accumulate credits towards credentials but the underlying educational experiences are often disjointed and fragmented (Kuh, 2001). Canning (1998) suggested that there is a need to bring coherence and structure to education through enhancement of the general education component of the curriculum. Relevance of the curriculum is another area of concern. Learners, employers, and the public look for abilities that will allow individuals to manage change more effectively in a competitive global community (Wingspread Group on Higher Education, 1993). Candy and Crebert (1991) analyzed the incontinuities between post-secondary education and the world of work and found that graduates were "often ill-equipped to deal with aspects of the workplace such as problem solving, decision making, working in a team, or learning for themselves" (p. 572). In their study of Canadian university students and graduates, Evers, Rush and Berdrow (1998) found that the skills in most demand by employers, visioning, creativity, risk taking and leadership, are also the areas in which students and graduates expressed the least confidence. Drake (1997) indicates that the demand for change in curriculum is mainly focused on the higher order thinking skills and the ability to interact with people. Gardiner (1994) argues along the same lines. We are not helping our learners to acquire the knowledge and abilities they will need to develop their careers or to fulfill their responsibilities as citizens. Martin (1985) supports the argument for integration, but from a slightly different perspective. She makes a distinction between productive and reproductive forces in defining  22  curriculum. She argues that our concept of being educated has focused solely on productive processes that relate to political, social, and economic aspects, but that we have failed to incorporate the reproductive aspects that relate to processes surrounding the nurturing of the family, thus leading to alienation of learners rather than their integration. These discussions about the nature and relevance of curricula are embedded in the ongoing debate about the aims and outcomes of education. Within the BC context Bauslaugh (1992) directs his critique towards general education and undergraduate programs. He suggests that our programs are "generally strong, but limited in their scope" (p. 5). They are strong in the sense that they provide the traditional discipline-based preparation for graduate school, specific careers and entry into professional schools, but limited in scope because they do not address the needs of learners who are not interested in pursuing graduate studies or entrance requirements into specific career programs. According to Bauslaugh the emphasis on general intellectual abilities, and an understanding of the ideas underlying our current global society are generally missing. This is not to suggest that the current experiences of learners in post-secondary education are not valuable both from economic and personal perspectives. The argument is that we are not meeting our potential in higher education. Many researchers contend that we need to support learners in developing their ability "to learn to learn" so that they can be prepared to adapt to a world we have yet to imagine (Candy & Crebert, 1991; Ministry of Employment and Immigration & Ministry of Industry, Science and Technology, 1991; Shugars et al., 1991; O'Neil, 1993; Wingspread Group on Higher Education, 1993; Gardiner, 1994; Evers et al., 1998). In the BC context, Gallagher (1995) suggests that the current structure and faculty approach in colleges focus on a role of dependency not independency. In summary, the calls for change are directed to student learning and the multiple elements that support that learning. The need for increased accountability and quality permeates the calls for change in higher education in Canada (Cutt & Dobell, 1998). However, this is not to suggest that there is agreement among educators and politicians about these issues. The literature suggests education has been in crisis for more than a century, and there is no general agreement as to the nature of the crisis or the issues that need to be addressed (Birnbaum & Shushok, 2001). Eisner (1994) suggests that the depth of analysis by the crisis advocates is shallow. Among the controversial and contested areas in my discussion are the issues of learner satisfaction  23  with higher education, the quality and quantity of research in education, and the relationship between education and work. Learner Satisfaction Based on data from the National Graduates Survey, Gingras et al. (2000) argue that student satisfaction levels have not changed over the past 20 years, and may in fact have increased with 82% of the 1990 Canadian university cohort reporting satisfaction with their education and job match compared to 71% of the 1982 cohort. They indicate that the data for vocational programs and career technical programs are similar. The authors of the BC strategic plan also state that their research indicates "that the public perceives the system as having provided a generally effective educational service to British Columbians over the years," and "approximately 70 percent of learners, employers and the general public support the system on the basis of the job it is doing" (BC MoEST, 1996, p. 14). The call for change is not necessarily based solely on the learners' or public's perceptions of higher education as often suggested by politicians. Type of Research When discussing educational research Cross (1999b) notes, "we know a lot about student learning. We know it through research and scholarship; we know it through our own experiences as learners; and we know it through the lessons our students teach us everyday" (p. 269). Cross proposes that we look for the gold nuggets found in previous research and melt them down into gold bullion. We have to integrate the insights that we have gained from practice in interpreting research results and we need to focus on more useful research that will shed more light on our practice worlds. This supports the earlier findings of Pascarella and Terenzini (1991) whose analysis of nearly 2600 studies of student learning indicates that research in education is substantial, and that we have considerable knowledge about the outcomes and value of post-secondary education. They found that college attendance was generally associated with significant gains in factual knowledge, and many abilities including critical thinking, analytical skills, and both verbal and written communication skills. Students also change with respect to self-esteem, values, attitudes and moral reasoning. The authors also argue that college attendance may generally have an effect in stabilizing abilities and preventing regression that may occur when abilities are not used. Rather than more research, we may just need different kinds of research to better understand learning.  24  Link Between Education and Work  The utilitarian link between education and work is also challenged in current educational debates. While the OECD (1997a) suggests that the return on educational investment for individuals is more obvious than the return for societies, Haveman and Wolfe (1994) found parental education to be an important positive factor in children's lives including an increased completion rate of high school, a decrease in nonmarital teenage pregnancies and a decrease in economic inactivity. However, Marginson (1997a) argues that the return on individual educational investment may also vary depending on the nature of the studies completed. Based on his analysis of British Columbian labour force data, Allen (1996) argues that university graduates have lower unemployment rates and higher earnings when compared to graduates of short-term or 2-year technical and vocational programs. Allen attributes this to the general employability skills that are woven throughout university programs. Gingras, Masse and Roy (2000) also suggest that occupations requiring high levels of communication and reasoning abilities have increased compared to other occupations; however, their overall numbers are small when compared to overall employment opportunities. These authors also did not find "any significant deterioration in the labour market situation of low-skilled workers relative to that of high-skilled workers" (p. 254). Avis (2000) questions the strength of the relationship between education and economic well-being beyond basic literacy and numeracy skills. However, projection of skill needs is a challenging venture given that employers may use education as a screening device (Rubenson, 1987) and given the anticipated skill shortages resulting in part from the aging of the current workforce (Gallagher & Lamoureux, 2001). Experiences in Canada and elsewhere indicated that the relationship between education and the economy was complex. An educational focus on employability does not necessarily translate into employment (Brown & Lauder, 1997). "Education cannot in itself generate capital movements or create wealth, except to the extent that it becomes a fullyfledged market commodity in its own right" (Marginson (1997, p. 29). Ainley (1998) and Avis (2000) go so far as to suggest that the relationship between the economy and education may be the reverse of what we currently perceive. The economy may stimulate further education, but education is only one of many factors in a complex array of variables that influence productivity and economic competitiveness; the importance of other complementary inputs and conditions should not be minimized (Levin & Kelly, 1997; Ainley,  25  1998). For example, Lipsey (2000) suggests that technological change is the "main engine of long term economic growth" (p. 47). While the focus on education is deemed to be important for a sustainable economy, its role may be exaggerated (Brown & Lauder, 1997; Woodhall, 1997; Levin & Kelley, 1997). Much of the rhetoric rests on an assumption of the relationship between competitiveness and the global economy; this is also the basis for much of the rhetoric about learning societies (Avis, 2000). Despite these arguments, human capital theory still appears to dominate the policy discourse surrounding higher education in British Columbia and elsewhere. Governments are interested in the degree of benefit to societiesfromthe investment in human capital (OECD, 1997a). Amid increased fiscal pressures, governments continue to look towards educational reform as a solution to economic problems. However, governments also want to know what they are getting for their investment in higher education (Brennan, 1997). Curt and Dobell (1988) indicate that Canadian taxpayers deserve a better explanation about the use of funds in post-secondary education. The state of the economy, the demand for more opportunities for higher education and concerns about the current status of higher education influence policy makers to assess the efficiency and effectiveness of higher education. "Meeting the costs of learning-thirsty societies will, in fact, be the question for the future. Given continuing constraints on public budgets and sluggish economic growth, there are no easy solutions" (Papadopoulos, 1998, p. 44). In the context of globalization and economic decline, issues of accountability and quality have become central themes for educational reform (Dunne, et al., 1997).  Key Words in Educational Reform Discussions about accountability, institutional management and cost-effectiveness permeate many public services, not just higher education. Neave (1988) suggests that these concerns and the movement to mass higher education led to the consolidation of previous evaluative initiatives and their reorientation to outcomes. This represents a shift in the traditional input-process-output and outcome model used in both public and private sectors. Such a shift aligns higher education more closely with national priorities. Fisher and Rubenson (1998) suggest that the accountability movement in Canada was influenced by a general suspicion of public institutions and a faith in thefreemarket. The following section presents an overview of the accountability movement that was at the heart of an evaluative  26  state, and the other key aspects associated with its implementation. The discussion revolves around the definition of accountability and its operationalization. Key words such as performance indicators, performance funding, efficiency and quality frame this discussion. Accountability A key theme in educational reform is accountability. Most educators would probably agree that accountability is an important and relevant discussion in higher education; it is an integral element of professionalism. Controversies about government accountability and organizational autonomy are not new, but Alexander (2000) suggests that the difference in current discussions of accountability is that they are based on the perception that "traditional measures of institutional performance and effectiveness such as peer review and market choice are not sufficient indicators of institutional value" (p. 414). The shift in the discussions implies a change in the acceptance of peer review but this has not been explicitly stated (Cave etal., 1991). Current discussions about accountability in higher education are directed to market accountability and political accountability (Ball, Vincent, & Radnor, 1997). Policy makers are interested in assessing the efficiency and quality of higher education in an attempt to make higher education more responsive to societal and economic demands. "Downsizing, . retrenchment and doing more with less have become themes, if not necessities, for most institutions" (Rush, 1995, p. 109). Educators are required "to do more with less," with the "more" adjective being directed to increased access and increased quality. In many jurisdictions accountability is operationalized in the form of performance indicators, which may then be linked to performance funding. Performance indicators have a long history (Kells, 1990; Bruneau & Savage, 2002), but they became prominent in the United Kingdom during the Thatcher years when they werefirstapplied to assess university research in Britain. This focus then shifted to teaching and learning as well. It was suggested that performance indicators would promote greater efficiency and quality in higher education (Cave etal., 1991). In the United States an emphasis on outcomes began in the mid-1980s driven by governments both at the national and state level. The National Governors Association was particularly influential in promoting the outcomes agenda (Manno, 1994). Themes were similar to those in the United Kingdom but were operationalized through the notion of assessment that was furthered by the accreditation system (Derlin et al., 1986). Accreditation  27  reviews shifted from a process focused on organizational improvement to one more directed towards monitoring external guidelines. Salvador (1996) suggests this represented a real change in the focus of accreditation and raised the fear that outcomes assessment would become the new orthodoxy of accreditation. However, Wright (2000) presents a different perspective. She contends that, "With proddingfromthe US Department of Education and the savvy cooperation of accreditors, assessment has proven to be an extraordinarily useful tool" (p. 54). She suggests that the assessment movement revitalized accreditation, and that the accreditation process in turn has kept the assessment movement alive. Whatever the relationship between accreditation and the assessment movement, there was an interaction between them. This dialogue about outcomes assessment in the United States is very similar to discussions in other parts of the world about performance indicators (Cuttance, Harman, Reynold, Macpherson, & Smart, 1998; Dale, 1999; Bruneau & Savage, 2002), but it was conducted in the context of self-regulation. Regardless of how the conversations were framed, the ultimate focus was directed to learner achievement, the outcomes of learning (Derlin etal., 1986). Productivity and efficiency were seen as the aims of accountability (Quinn & Rohrbaugh, 1983). However, as Bruneau and Savage (2002) argue "efficiency is not an end it itself - one must ask, efficiency in aid of what goals?" (p. 11). Discussions about efficiency stimulated conversations about quality and equity (Creemers, 1997), and while both efficiency and quality are elusive and highly contested constructs, performance indicators have been constructed to measure them.  Performance Indicators Performance indicators are by no means a new phenomenon. Bruneau and Savage (2002) trace the history of performance indicators to the nineteenth and twentieth century although they suggest that traces of such measurements are evident as far back as the renaissance period. They identifyfivephases beginning in 1850: the origins phase, the efficiency phase, the behaviourist and accountancy phase , the systems theory and 1  management phase, and lastly the performance indicator phase that came into vogue with the election of more conservative governments in the United Kingdom and the United States in the 1980s.  This phase also included the influences from the electronic sciences and industry production systems.  28  The OECD was influential in developing indicators to support public accountability in education (OECD, 1998a). The Centre for Educational Research and Innovation (CERI) directed its attention to educational indicators in 1988 (OECD, 1995a). These indicators were designed to provide input to national policy makers about monitoring systems and educational systems in general. They developed a framework that included demographic and social characteristics of the population (context and inputs), educational programs and processes (process) and outcomes (outputs). However they cautioned against interpreting the relationship between elements of the framework as causal relationships, describing it instead as a conceptual map, not a model. As suggested by the OECD approach, indicators are frequently based on some combination of inputs, processes and outputs or outcomes (Graney & Kellaghan, 1996). However the current shift towards an evaluative state places increasing emphasis on outcomes, and the relationship between inputs and outcomes or outputs . While the 2  development of performance indicators involves some rocky terrain, there are common approaches used to define the outcomes of teaching and learning (Derlin et al., 1986; Cave et al., 1991; Cave, Hanney, Henkel, & Kogan, 1997). These include the following: cost-benefit approach, cost-effectiveness approach, and valued-added approach. These approaches are embedded in the constructs of efficiency and quality. Performance indicators are used to measure the efficiency and quality of higher education and through this measurement process is postulated to support the overall accountability of higher education. Supporters and opponents of performance indicators agree that the landscape of indicators is a challenging terrain. Strathern (2000) points to issues surrounding validity and reliability of the data and raises the concern that "visibility as a conduit for knowledge is elided with visibility as an instrument for control" (p. 311). Politicians and administrators are frequently influenced by cost-effectiveness so there is a tendency to use data at hand and data that can be easily gathered (Cave et al., 1991; Bruneau & Savage, 2002). In their study of institutional assessment practices, Peterson and Einarson (2001) found that organizations tended to focus on data that could be easily quantified such as employment outcomes and further education. More complex measures such as cognitive development were not used as frequently. Yorke (2000) uses the analogy of a coconut to describe the risks associated with performance indicators. The terms "outcomes" and "outputs" are used synonymously by some and are differentiated by others. Those that differentiate view outcomes as being related to quality issues, and outputs as attrition and graduation data (Caveetal., 1991). 2  29  The information available about institutional performance has the characteristics of a coconut - a hard shell surrounding a softer center. The danger is that attention will focus upon hard data that can be measured with reasonable accuracy and that less weight will be given to the softer kinds of judgment about what is important, (p. 2) Measurements tend to become more difficult and less reliable as complexity of the unit to be assessed increases (Karmel, 1996). Often the indicators we wish to measure may be too impractical and costly to implement. However, using a smorgasbord of data from previous studies is also problematic. There is a general concern about the adequacy of the information systems needed to sustain performance indicators and their management (Kells, 1990; Bruneau & Savage, 2002). This concern is heightened in times of fiscal restraint, and indicators are often criticized for being "highly constructed and artificial means of measuring real output" (Strathern, 2000, p. 311). In attempting to meet the demands for public accountability, policy makers and administrators may be confounding the construct of quality through the introduction of unrelated measures. This is particularly relevant to postsecondary education whose aims are more complex when compared to the business sector that commonly has more clearly defined and simple goals such as maximizing returns on investment (Karmel, 1996). As well as the measurement issue, there are also interpretation challenges. "Because of their partial nature, individual performance indicators often provide potentially misleading impressions even of average productivity" (Cave et al., 1991, p. 34). Writing from a BC perspective in K-12 context, Sullivan (1988) indicated that many variables needed to be measured to gain an understanding of learner outcomes. He suggested it would be "perverse to establish and act on the basis of performance measures which failed to capture the primary objectives of the activity measured" (p. 181). In the case of subject assessment Bruneau and Savage (2002) suggest that interpretation of performance indicators is at least as challenging as facilitating the learning itself. The issue of interpretation is closely linked with concerns about the use of performance indicators. "Whilst information may be innocent, the use to which it is put may not be" (Kells, 1990, p. 7). Initially the drive was to find performance indicators that could be used to compare the outcomes of organizations within systems (Cave et al., 1991). There were concerns that a comparative approach would lead to organizational ranking (Kells, 1990); this of course would disadvantage organizations with less resources. Yorke (2000)  30  identified the potentially punitive aspects of performance indicators, arguing that we need to anticipate their possible effects and their side effects. Indicators should also to be within the control of educators. Concern is expressed that post-secondary organizations will be held accountable for economic and social variables over which they have no control (Featherman,1993; Bruneau & Savage, 2002). Some researchers claim that governments and the public have unrealistic expectations of postsecondary education (Kenny-Wallace, 1988) fueled by a desire to see change within one election period (Webber & Townsend, 1998). The relationship between the performance indicators and the responsibilities of higher education need to be clarified. Educators are also concerned about the relationship between performance indicators and funding. This discussion is sometimes euphemistically couched in terms of "incentives for performance" (Seppanen, 1998) or "funding mechanisms to reward performance" (Hildebrand, 1998). This is a particular concern in British Columbia as two of its neighbours, the state of Washington and the province of Alberta have initiated links between performance indicators and funding. Ultimately the discussions about outcomes and performance indicators often become linked to performance funding (Kells, 1990; Bruneau & Savage, 2002). Despite the enormous costs and challenges associated with the implementation of performance indicators in the United Kingdom, New Zealand and Australia, advocates of reform continue to promote their implementation (Collins, 1993; Beevers, 1993; Ecclestone, 1994; Bruneau & Savage, 2002). The audit culture is ubiquitous in the public sector, and particularly so in higher education (Jackson, 1993; Foley, 1999).  Efficiency and Quality Discussions about accountability often focus on the measurement of efficiency and quality. Quinn and Rohrbaugh (1983) discuss the challenges of this narrowly focused dialogue. Efficiency is not a concept that can be easily observed or measured; it is a complex construct. Agreement has not been achieved on the actual concepts to be included within it. The same applies to quality; it is an elusive and highly contested construct. It is a frequently used but often ill-defined term (Karmel, 1996). Historically the concept of quality was associated with ideas of excellence or outstanding performance. Now it is more closely linked to the ideas of efficiency and effectiveness. The construct of efficiency has been subsumed by the broader construct of quality (Cave et al., 1997).  31  The construct of educational quality has become embedded in the language of business management. It often reflects cost cutting and increased competition (Brennan, 1997). The assessment of quality came into vogue through quality assurance, quality audits, and total quality management approaches, and this theme continues to evolve (Joss & Kogan, 1995). The most recent version is the idea of continuous quality improvement. These approaches to quality have created tensions within higher education. The academic perspectives on quality focus on standards, coherence and understanding (Brennan, 1997), and are often at odds with efficiency models drawn from the private sector. Many authors agree that quality is an elusive, complex and multi-faceted concept (Derlin et al., 1986; Nadeau 1992; Dennison, 1995; Woodhouse, 1999). Attempts to define quality have ranged from philosophical approaches to operational definitions based on many different variables such as faculty credentials, number of hours, course work, grades, and employment. Quality is often approached from several perspectives including reputational, resources, outcomes and value-added perspectives (Derlin et al., 1986; Dennison, 1995). Gaining general consensus on quality indicators or attributes may be a manageable task. Nadeau (1995) developed a list of indicators reflecting the classic inputs, process, context, and outputs / outcomes model. He conducted a study using a modified FOCUSDELPHI technique to determine the perceptions of major stakeholders in education regarding the indicators and criteria related to the concept of quality and excellence. During the final validation phase, 1,113 (77%) of the 1,447 indicators of quality and excellence were rated as 3.5 or above on a five-point scale related to their importance. Except in criteria related to research, no differences were found by region, by language spoken, or by type of organization (i.e., colleges and universities). However, Quinn and Rohrbaugh (1983) note the limitations of using multivariate analysis in defining the construct of quality. The outcome is ultimately dependent on the initial task of selecting the measures; regardless of the approach used in the development of the measures there are inherent risks of bias embedded in their selection. If nothing else, Nadeau's analysis supports the notion that the evaluation of quality is a complex venture. However, a list of indicators may not bring us closer to the issue of educational quality in the absence of specific measurements and it may not be possible to measure quality independently. While focusing on process may be limiting, the same argument could be made for the focus on outcomes. Afshar (1990) suggests that a more comprehensive approach to quality may be reached through applying the Attributive Theory of Quality, which he defines as "the  32  interactive sum of all the necessary and sufficient properties that comprise a phenomenon" (p. 12). Others also suggest a broader perspective, a systems approach to measuring quality (Dahllof, 1991; Cavanaugh, 1993; Lewis & Smith, 1994). The process of evaluation becomes a tool for continuous improvement in education. The approach of building quality elements into the system and constantly monitoring the system is attributed to the work of W. Edwards Deming and other authors of total quality management in business (Lewis & Smith, 1994). The actual terms used in such systems models may vary slightly but the underlying concept is the same. They acknowledge the complexity of assessing quality, and the need to look at the relationship between elements within the system to gain a more meaningful understanding of the quality phenomenon. Cave et al. (1997) suggest that performance indicators may have a role to play in overall quality management. They may provide a context for decision-making, and may help to determine certain key issues or questions to address. This change to a systems approach reflects a further change in the concept of quality, one that suggests a negotiation between excellence and efficiency. The most commonly accepted definition appears to be "fitness for purpose" (Woodhouse, 1990; Stephenson, 1998). Institutions are evaluated to determine how well they meet their defined mission and purpose. This allows for organizational diversity while still meeting the needs for accountability. But as Stephenson stresses, this approach often takes the purpose of education as a given. This fitness for purpose is the focus on quality that is currently applied by the commissions on accreditation in the United States. Wright (2000) suggests that the increased focus by the American accreditation commissions on the assessment movement has changed the definition of quality from a reputational and resources perspective to one focused on learning outcomes, development / improvement, or value-added approaches. She suggests that this shift has made educational quality "more inclusive, more democratic, more egalitarian" (p. 55). This echoes the views of Dennison (1995) who suggests that the concept of talent development may provide opportunities for organizations that work more closely with marginalized groups to demonstrate their contribution to the social and economic benefits of its learners. The Maritime Provinces Higher Education Commission (MPHEC) has adopted this type of broad definition of quality. "First and foremost, quality is directly linked to, and can mostly be measured by, the extent to which each institution realizes its mission" (MPHEC,  1997a, p. 1). The BC institutional evaluation process is based on a similar approach to measuring quality (AECBC, 1991). This approach integrates the new emphasis on outcomes with more historical approaches to quality. Assessing the effectiveness and quality of an organization or a system ultimately involves the question of values. Quinn and Rohrbaugh (1983) identify this as one of the major problems in this area. The pertinent values have never been clearly articulated. This is a reflection of the ongoing debate about the aims of education, an issue we have been grappling with for over 2000 years. Dennison (1995) has labeled the quest for accountability in higher education as a mission impossible. "Accountability is in the eye of the stakeholder, each of which may demand different services, performances, and outcomes. While there may be superficial agreement upon performance indicators, they are usually so impractical that they cannot be measured" (Dennison, 1995, p. 241). Accountability appears to be a term understood by many, yet its definition and measurement is disputed and controversial. The same applies to the construct of efficiency and quality; the highly abstract nature of these constructs and the lack of agreement regarding their analysis may account for some of the confusion and ambiguity in the literature. Brennan (1997) suggests that the controversies surrounding quality are based in language and power issues. The dialogue is couched in the market language of targets and performance indicators, but they are suffused with ideas of social inclusion, co-operation and partnerships (Avis, 2000). Governments tend to defined quality from a consumer perspective (Cave et al., 1997). Competition and teamwork are intermixed in the rhetoric. However, the imposition of corporate language threatens the autonomy of higher education (Brennan, 1997). This analysis of the accountability movement and its associated performance indicator measurements sets the stage for the discussions surrounding learning and the outcomes of learning.  34  Learning Outcomes Approach The relationship between education and the economy is at the core of educational reform initiatives, and the development of learning outcomes are described as an important aspect of this relationship (BC MoEST, 1996). A learning outcomes approach is viewed as a way to promote enhanced learning and increased accountability (Drake, 1997). In particular, discussions about learning outcomes focus on the relationship between education and work (Brennan, Kogan, & Teichler, 1996; Betcherman, McMullen, & Davidman, 1998; Fisher & Rubenson, 1998). Learning outcomes are directed towards the abilities that graduates need to assume roles in society and the economy in particular. Through the strategic plan (MoEST, 1996), C2T2 was directed to promote a learning outcomes approach. This initiative was introduced amidst discussions of quality, efficiency and accountability that were also embedded in the strategic plan. One challenging aspect of the learning outcomes initiative in British Columbia was to define it. What exactly was a learning outcomes approach? What if anything was new or different about this approach? At first glance, these appeared to be simple questions but they formed the basis of lengthy debates in British Columbia. In this section I analyze the origins of the learning outcomes debates, define a learning outcomes approach, discuss its relationship to educational reform, and review the evidence to support its position as a reform initiative.  Origins of Learning Outcomes Discussions about learning outcomes originate from several sources, although the influences ultimately blend in conversations about curriculum reform. The previous sections identified the political origins of the conversations about the "outcomes of learning" defined in multiple ways. From this perspective, performance indicators would be developed to measure the outcomes of learning. Certain outcomes measures would be directed to the abilities that learners possess as a result of their educational experiences. Defining and measuring "learning outcomes" would thus support accountability, efficiency and quality in higher education. Discussions about learning outcomes also arise from a pedagogical perspective. Lazerson et al. (2000) identify the development of a movement to take teaching and learning seriously. They suggest this movement runs parallel to the assessment movement. From this perspective learning outcomes are linked to notions of coherence (Mentkowski, 1998), and  35  clarity and transparency (Avis, 2000). They also focus on issues of relevance (Candy & Crebert, 1991; Battersby, 1999), a term that is often connected to the construct of quality (Derlin et al., 1986; Newman, 1999). Many of these discussions are linked by a concept called "integrated education," a term described by Jennings (1997) in the following way: "integrative education is defined as education that promotes learning and teaching in nonfragmented ways that embrace notions of holism, complexity, and interconnection. Integrative education rejects the common emphasis on transmitted knowledge" (p. 2). Such an approach is described as embracing the links rather than the differences between disciplines. Discussions about curriculum reform are often couched in outcomes language and focus on the abilities that would support graduates in a post-industrial labour market (Carmichael, 1993; Betcherman et al., 1998). An increased emphasis on assessment of learning brought the economic and pedagogical elements together. Models were built around common outcomes for student performance (Haworth & Browne, 1992; Drake, 1997; McDaniel, Felder, Gordon, Hrutka, & Quinn, 2000), and performances were assessed to determine student learning. Performance auditing was accepted as a new type of professional conduct (Barzelay, 1997), and assessment of learning became a focus for promoting learning as well as demonstrating accountability (Schmitz, 1994). The previous sections focused on the political discussions surrounding the outcomes of learning. In the next sections I focus primarily on the pedagogical discussions that shaped the concept of learning outcomes.  Learning Outcomes  Defined  The conversations in British Columbia revolve around the outcomes of learning and learning outcomes. Some use the terms interchangeably while others discuss the idea of learning outcomes as one aspect of the larger conversation about the outcomes of learning. The terms also merge with conversations about competency based education (CBE), outcome based education (OBE), skills and abilities. This multiplicity of terms is not unusual in policy texts (Bowe, Ball, & Gold, 1992). It is an example of how concepts are shaped in practice by educators and politicians. It seems logical to define what you expect students to learn, and then to design the instructional activities and the evaluation structure around those defined learner outcomes. Educators have been discussing outcomes for many years in relationship to course goals,  36  behavioural objectives, instructional objectives and competencies. The literature is replete with approaches to defining educational intention including the following: educational objectives (Tyler, 1949), instructional objectives (Mager, 1975), behavioural and non behavioural objectives (Cohen, Manion, & Morrison, 1996), and expressive objectives (Eisner, 1979). Taxonomies of educational intentions were part of the movement to bring clarity to curriculum and evaluation. Bloom's taxonomy of Educational Objectives (1956) was the most influential of these, and it linked well with the work of Mager (1975) related to instructional objectives. While Bloom's taxonomy was very influential (Anderson & Sosniak, 1994), it was also criticized for its simplistic and hierarchical approach (Marzano, 2001). The 1980s saw a shift in emphasis towards higher order thinking and reasoning abilities. New taxonomies emerged. The Structure of the Observed Learning Outcomes (SOLO) Taxonomy (Biggs & Collis, 1982) focused on the quality of learning by analyzing the structure of an individual's response. Mezirow (1998) developed a Taxonomy of Critical Reflection Assumptions to assist educators in facilitating self-assessment. Jans and Leclercq (1997) argue along a similar vein in proposing the need for a taxonomy of metacognitive realism to assist learners in the development of self-assessment abilities. Ultimately these taxonomies were designed to provide ways of understanding learning and the evaluation of learning. As is evident from the previous discussions, we have a plethora of terms and definitions related to the purpose or aims of education. Allan (1996) suggests that our situation "arises from the liberal use of a number of labels to connote statements of purpose which operate at different levels of specificity, with the result that the literature of educational intention has become a minefield of terminological confusion" (p. 93). This is not necessarily a recent phenomenon but it may have been accentuated by the introduction of yet another way of describing our educational intentions and outcomes. The concept of learning outcomes is generally defined in very broad terms. The following list provides examples of definitionsfroma Canadian context: Learning outcomes represent the integration of knowledge, concepts, skills and dispositions in complex role performances. (Shipley, 1995, p. 13) The learning outcomes approach means basing program and curriculum design, content and delivery on an identification of the knowledge skills and values needed by both students and society. ... Learning outcomes are thus the knowledge, skills and values acquired by students as a result of their educational experiences . (Bauslaugh & Hansen, 1996, p. E2-E3) 3  3  The bolded text reflects the authors' emphasis.  37  Learning outcomes are statements of the results of the learning process. They identify what the learner is able to do or perform as a result of their learning experience. They may include statements about the learner's knowledge, skills, abilities and values. (Stanley & Mason, 1997, p. 5) Learning outcomes represent culminating demonstrations of learning and achievement. They are not simply a listing of discrete skills, nor broad statement of knowledge and comprehension. They describe performances that demonstrate that significant learning has been verified and achieved by graduates of the program. (College Standards and Accreditation Council, 1995, p. 2) These policy texts highlight a variety of themes connected to the concept of learning outcomes. Outcomes based education falls within the area of "knowing how" as opposed to "knowing that" (Hutmacher, 1997). Advocates of reform use the term learning outcome to refer to a specific approach for documenting the purpose of education from a learners' perspective. The definitions place an emphasis on the proof of outcomes, on demonstrations of learning, and suggest that these demonstrations focus on significant outcomes reflecting an authentic environment. Learning outcomes are defined in broad, general terms so as to reflect cumulative learning and include the notion of knowledge, skills and attitudes, language that is reminiscent of Bloom's taxonomy (Bloom, 1956). They are described as being nondisciplinary and transferable (Dunne et al., 1997). Learning is also intended to be reflective of the "real world" but this characteristic is often slanted towards an economic and employment focus (Curtain & Hayton, 1993; Ecclestone, 1994 ). Much of the literature addresses the notion of graduate outcomes, but in the BC literature there is also discussion about the outcomes being adapted to reflect the outcomes of specific courses (Bauslaugh & Hansen, 1996; Battersby, 1999). This is evident in the approach taken by the faculty members at Alverno College; their abilities are translated into course outcomes. The broader outcomes are threaded throughout the curriculum permeating each course. This is often described as designing down the curriculum (Spady, 1994; Schmitz, 1994; Drake, 1997). Proponents attempt to distinguish between learning outcomes and other ways of expressing the outcomes of learning. Shipley (1995) presents a dichotomous approach in which the competency and behavioural approach is indicated as being narrow in focus, employment oriented, and content driven. In contrast learning outcomes are described as learner-centered, integrated, transferable, and related to adult life and work (see Table 1). Others present the difference by degree of emphasis. Learning outcomes are proposed to be  38  more integrated and more holistic (Battersby, 1999). Learning outcomes are described as being a product of systems theory and education is regarded as a learning system (Bauslaugh 4  & Hansen, 1996). The concept of systems theory is attributed to the biologist Ludwig von Bertalanffy in the 1960s (Laszlo, 1972a). He emphasized the organization of elements and their interdependence. A system was defined as an organized whole with boundaries that were open and through which energy was exchanged (Potts & Hagan, 2000). The concept was originally applied to sciences but developed interdisciplinary acceptance as it was shaped into the concept of'systems approach' and 'systems thinking' (Laszlo, 1994). However, the acceptance of this concept was not universal. Opponents viewed it as simplistic, mechanistic and the imposition of a worldview based on positivism and behaviourism (Lilienfeld, 1978). Despite its opponents the concept continues to be discussed as a way of focusing on holism and integration (Skyttner, 2001). Its application to learning outcomes appears to be focused towards the integration of elements surrounding learning thus supporting holistic learning experiences and outcomes. The distinctions made by the advocates of a learning outcomes approach are not as obvious as they wish them to be. In British Columbia the DACUM (Develop a Curriculum) process has been popular in applied areas. This approach is used to define exit competencies described in terms of the skills, knowledge and attitudes graduates require to integrate into their respective practice environments. Joyner (1995) suggests that the DACUM approach and CBE are often considered as one concept or process. However, he distinguishes between the two, suggesting that the focus of competencies is on 'how' students learn while the DACUM approach addresses 'what' they should learn. This may have been true of the original competency format which had an underlying assumption that following the process guidelines in sequential order would lead to the defined outcome (Sunell, 1998). However it may no longer be valid given the evolution of competency frameworks in recent years. As educators worked with the competency framework, it evolved from descriptions of discrete technical tasks to explanations of complex exit skills. Reynolds and Salters (1995) suggest that several competency models have emerged, with the first ones focusing on behaviour at the cost of knowledge and understanding. Further models adopted a more holistic approach to include additional elements affecting performance such as understanding, knowledge and values. Competencies are described in terms of a "general capability based on  4  Ruth Steihl from the University of Oregon also used this language during her workshops in BC.  39  Table 1.  How are Learning Outcomes Different from Behavioral Objectives / Competencies Learning Outcomes  NOT  performance specifications which describe performances demonstrated in authentic contexts.  not  adult life / work role expectations  not  job-specific skills / tasks / knowledge  essential outcomes which represent exit standards for a program / course / unit of learning  not  preferred outputs which are demonstrated in sequence and measured at specific intervals throughout the course.  the results of integrated learning (knowledge / concepts / skills / dispositions) expressed as role performances.  not  intentions that drive curriculum design  transferable abilities based on integrated learning applicable in many contexts  not  directly observable behaviors that are specific to context, content, conditions and time  learner-centered and performancebased  not  discipline / subject-centered or content based.  Behavioral Objectives / Competencies design specifications which describe inputs such as topics to be covered or discrete skills to be mastered during the course.  From Shipley, 1995, p. 17  knowledge, experience, values, dispositions which a person has developed through involvement with educational practices" (Hutmacher, 1997, p. 45). In Australia professional and paraprofessional programs use a broad and holistic approach in defining their competencies (Curtain & Hayton, 1993). The same approach is evident in North America. Chambers and Gerrow (1994) point out that in dentistry the term competency is most often used to "describe the skills, understanding and professional values of an individual ready for beginning independent dental or allied oral health care practice" (p.361). This definition suggests that the distinction between current competency frameworks and learning outcomes is minimal. In Australia the term "key competencies" is used to define broad statements of ability (Haworth, & Browne, 1992); in the United States the term "necessary skills" and "core competencies" are used to describe similar abilities (Secretary's Commission on Achieving Necessary Skills, 1992; Wilson et al., 2000). The competency  40  based education movement represented a shift away from a disciplines oriented education model, to an outcomes approach, a movement that is still evolving. Drake (1997) describes this evolution as a movement from the behaviourist approach of the 1960s to a constructivist approach in which "being able to do" suggests that learners have really learned something. Faculty members at Alverno College use the term abilities, which they describe as "multidimensional, as complex combinations of skills, self-perceptions, attitudes, values, knowledge, and behaviors" (Mentkowski, 1990, p. 3). The Alverno faculty base their curriculum and assessment on eight abilities that are threaded throughout their diploma, degree and post-graduate programs. These abilities are similar to the "key competencies" developed in Australia (Haworth & Browne, 1992), and the "necessary skills" developed by the American Department of Labor (Secretary's Commission on Achieving Necessary Skills, 1992). While these terms appear to encompass similar notions, skills and competencies have historically been associated with occupations and the trades in particular; they are not widely associated with non-vocational undergraduate studies (Hodgson, Spours, & Savouy, 2001). 5  The difficulty in classifying the abilities developed in non-vocational programs may have influenced the development of additional terms relating to outcomes of learning. Allan (1996) refers to "personal outcomes" that are subdivided into "personal transferable skills" and "generic academic outcomes." These are deemed to typify graduates abilities but are thought to transcend specific disciplinary boundaries. Stephenson (1998) frames similar discussions in terms of "capability." Capability embraces competence but is also forward-looking, concerned with the realization of potential. A capability approach focuses on the capacity of individuals to participate in the formulation of their own developmental needs and those of the context in which they work and live. A capability approach is developmental and is driven essentially by all the participants based on their capacity to manage their own learning, and their proven ability to bring about change in both. (Stephenson, 1998, p.3) This term is similar to the idea of transferable personal skills. Allan (1996) argues that the notion of transferability is not specifically embedded in the concept of capability, but the language used by Stephenson seems to suggest its inclusion. The concept of learning outcomes is not new, but rather a re-shaping of an old concept. As Howlett and Ramesh (1995) suggest, most policies do not "have a definite life cycle - moving from birth to death - but rather seem to recur, in slightly different guises" (p. 10). Educators moved from the more general educational objective to more discrete 5  The health professions in North America and the professions in Australia are an exception to this.  41  objectives such as instructional objectives and behavioural objectives. The pendulum is now swinging back. From this perspective the current focus on learning outcomes reflects a shift along a continuum, from narrow and specific to general and holistic outcome statements. However, she does point out one new twist. She suggests that with the learning outcomes approach there is no assumption that the outcomes are related to the teaching and the course itself. This acknowledges the role of the learner and the fact that learning can occur in many places. From this perspective the term has the potential for wider application and includes educational and credentialing elements. However, the literature from Australia and the United Kingdom suggests that current competency frameworks are also based on this assumption (Curtain & Hayton, 1993; Fitzsimons, 1999; Yorke, 2000). Regardless of the term applied, discussions about the outcomes of learning focus on what learners "know," "value" and are "able to do." The outcomes are described in terms of complex abilities that are multidimensional as opposed to simple, unitary constructs (Mentkowski, Astin, Ewell, & Moran, 1991). In the conversations between these authors in the previously cited reference, Mentkowski describes the nuances of this current approach to abilities: But those abilities are more than multidimensional; they're holistic. They include qualities of the person. They include not just knowledge or skills but attitudes, behaviors, even dispositions. We're beginning to understand that something like critical thinking has cognitive, affective, social, even kinesthetic dimensions. Moreover, we define those abilities as transferable and we expect them to last a lifetime, to transfer across multiple aspects of work, family and civic life long after college, (p. 13) Spady (1994) suggests outcomes that are called transformative. As Drake (1997) identifies, these approaches take us beyond the notion of "doing," to the concept of "being." It shifts the discussions to the development of the whole person. While educators may shape the concept of learning outcomes along these personal development themes, policy makers tend to focus more narrowly along employability themes. There are often confusions and tensions created by these different approaches (Manno, 1994). Researchers may attempt to differentiate between ways of expressing the outcomes of learning but the various concepts are often blended in the world of practice. A recent study of members of the League for Innovation in the Community College (Wilson et al., 2000) looked at this issue of language by asking administrators to identify the terms most often used by faculty to refer to so-called 2l Century Skills. The most commonly selected term was st  42  general education core (34%) and core competencies (26%) (see Table 2). The term, 21  st  Century Skills, coined by the authors for these learning outcomes was used by only 3% of the respondents. The terms that respondents recorded in the 'other' category included core abilities, and general education and workplace competencies. The term learning outcomes was not included in the list of responses, but the authors use the term as an overarching construct for the plethora of terms that have been coined to describe the outcomes of education. Dunne et al., (1997) found similar results in their study of British educators. Regardless of the influences affecting the choice of term, the process by which terms are adopted is an example of how policy texts are shaped in the world of practice. Table 2.  Terms Used to Refer to the 21 Century Skills* st  Terms 21 Century Skills Basic Skills Core Competencies Core Skills General Education Core Generic Skills Life or Critical Life Skills Work Skills Other Total st  Number  Percent  8 21 62 10 81 8 8 14 30 242  3% 9% 26% 4% 34% 3% 3% 6% 12%  -  * the respondents were requested to check one item "that faculty and staff used most often when referring to 21 Century Skills" (Wilson et al., 2000, p. 19). st  Although consensus has not been achieved regarding the specific terminology to be used, analysis of literature indicates there is some agreement about the general abilities required to live and work in a world of constant change. In Canada the Conference Board of Canada (1992) was influential in articulating the perceived needs associated with the employment sector (see Appendix A). It developed the Employability Skills Profile that includes academic skills, personal management skills and teamwork skills. A similar skill set was identified in the report Learning Well... Living Well, a consultation paper through the Ministry of Employment and Immigration & Ministry of Industry, Science and Technology (1991). The following are described as the basic skills required: • •  the ability to learn, the most basic skill of all; reading, writing and computation skills;  43  • • • • •  oral communication and listening skills; problem solving and creative thinking; skills and values needed to achieve high self-esteem, motivation and goal setting; employability and career development skills; interpersonal, teamwork and negotiation skills, and skills related to understanding organizational culture and the sharing of leadership. (1991, p.14)  In a more recent study of Canadian university students and graduates, Evers et al. (1998) identified four competencies required in today's work place: managing self, communicating, managing people and tasks, and mobilizing innovation and change. The abilities reported in the Canadian literature are similar to ones recorded in international documents from the United Kingdom (Hodgson et al., 2001), Australia (Queensland Department of Education, & Queensland Vocational Education, Training and Employment Commission, 1994 ), New Zealand (New Zealand Qualifications Authority, 1994) the United States (Secretary's Commission on Achieving Necessary Skills, 1992; Schmitz, 1994; Wilson et al., 2000) and Europe (Hutmacher, 1997). An analysis of these documents (see Appendix B) suggests that all have the following abilities in common:. •  communication (oral, written, technology);  •  interpersonal abilities (working with others);  •  thinking and problem solving;  •  managing self (responsibility, ethical approach, flexibility, adaptability); and  •  ability to learn independently (accessing information, numeric literacy, computer use, reading and writing).  These abilities reflect the focus on the development of human capital to support economic prosperity. Others focused on the need to educate for citizenship (Atwell, 1993; Spady, 1994; Usher et al., 1997). Atwell (1993) makes a distinction between what society wants from higher education, and what society needs. Society's wants tend to be more instrumental in nature, often associated with economic considerations and the world of work. However, he believes what society needs from higher education is "a set of interrelated roles and functions: the teaching of citizenship and values; the academy as an independent critic of society; and higher education as an agent of social change" (p. 51). He suggests these are essential for society to manage complex economic, political and social questions in an effective and humane manner. "The job of the colleges and universities, then, is to prepare  44  students to be citizens who can make wise choices and exercise leadership in all spheres of society" (p. 51). This notion of participatory citizenship is sometimes labeled as community service, service and social responsibility. Such a direction is the basis of the approach to education described in the UNESCO document Learning, The Treasure Within (Delors, 1996). Four pillars are described as the foundations for education: learning to know, learning to do, learning to be, and learning to live together. Thefirstthree pillars are seen as supports for the fourth pillar, learning to live together. The UNESCO focus draws the economic and social issues together. A similar theme was expressed earlier by Bosworth (1993). "We should not concentrate so exclusively on teaching young Americans how to work in an increasingly complex, technologically driven world, that we neglect to teach them how to live in such a world" (p. 57-58). The discussions about learning outcomes were, however, primarily stimulated by arguments that a skill gap existed between the abilities of learners and the needs of employers. This gap was perceived to be affecting economic prosperity (Reynolds & Salters, 1995; Greaney & Kellaghan, 1996; Ainley, 1998). In the United Kingdom, Australia and New Zealand this led to the development of national standards or qualification frameworks. The frameworks were designed to integrate general and vocational education, but in practice the focus was largely directed to vocational education (Curtain & Hayton, 1993). The discussions in the United States and Canada tended to be more generic and eclectic. In the United States the focus was largely directed to the implementation of the SCANS competencies and foundation skills (Secretary's Commission on Achieving Necessary Skills, 1992). In Canada the dialogue centered around the development of national standards for labour mobility (Human Resources Development Canada, 1994). Despite these differences, the debate about the value of a learning outcomes approach remains a contested issue among educators in these jurisdictions.  Benefits and Limitations of a Learning Outcomes Approach Both the proponents and the opponents of a learning outcomes approach are passionate in their discussions of the benefits and limitations of such an approach. In this section, I will review these perspectives. The benefits of a learning outcomes approach are described in terms of supporting learning as well as providing instrumental use that allows for the recognition of prior learning, for increased communication among those with an interest in education, and for the demonstration of accountability to policy makers and the  45  public (Mentkowski, 1998; Canning, 1998; Candy & Crebert, 1991; Kuh, 2002). The opponents of learning outcomes question the ability of this approach to deliver these benefits (Collins, 1993, Jackson, 1993; Ryan, 1998; Avis, 2000; Strathern, 2000). In fact they argue that it makes education the handmaiden of capitalism and addresses managerial needs, rather than learning needs. Perceived Benefits A major aspect of the discussion surrounding the benefit of a learning outcomes approach rests on its ability to support learning relevant to global societies. The arguments are frequently grounded in concerns about a mismatch between what learners are gaining from education and the abilities they will need for work and citizenship (Reynolds & Salters, 1995; Greaney & Kallaghan, 1996; Dunne et al., 1997). A main theme in the debate is the idea of bringing coherence and structure to education (Jennings, 1997; Mentkowski, 1998; Canning, 1998; Kuh, 2002). This argument rests on the premise that traditional disciplinary approaches have tended to fragment curricula in ways that may no longer be relevant in our knowledge society. The search for coherence arisesfromthe charge that we have not articulated our core values and goals (Drake, 1997), and that the relationship between the goals of liberal arts education and our teaching and assessment methods are not symbiotic (Mentkowski, 1998; Papadopoulos, 1998). Knowledge has become separated from its implementation; experiential learning needs to be acknowledged along side cognitive learning. Candy (2000) suggests that abilities are one way of promoting vertical integration of the curriculum and helping educators manage the "information overdose" that is a reflection of the exponential increase in knowledge. A key element of coherence in education as discussed in the United States, is connecting assessment with learning (Loacker & Mentkowski, 1994). Abilities-based education challenges educators to reassess existing assessment strategies (Ecclestone, 1994). Current approaches to assessment may not be meaningful when the aims of education are intellectual, moral and personal development (Mentkowski, 1990). From this perspective an outcomes approach promotes a realignment of the curriculum, implementation and assessment strategies to harmonize these elements. As well as coherence, learning outcomes are also perceived as promoting clarity and transparency (Ecclestone, 1994; Wilde & Hardaker, 1997). From this perspective a learning outcomes approach reflects a shift in power and language. Learning becomes a public,  46  explicit and shared experience (McDaniel et al., 2000). "The concern is to render transparent the 'secret garden' of curricula and assessment" (Avis, 2000, p. 41). This transparency is expected to assist non-traditional learners to better understand what is expected of them. From this perspective learning outcomes promote access and success (Ecclestone, 1994; McDaniel et al., 2000). Learning outcomes are also viewed as a way of promoting relevance of the curriculum and its associated learning experiences. Discussions revolve around the idea of bridging the gap between general and vocational education (Collins, 1993; Ecclestone, 1994). A focus on abilities allows educators to meet the needs of employers and society as "the kinds of complex thinking, communication, and problem solving skills learned in college are in great demand in the workplace as well as one's personal life" (Mentkowski, 1991, p. 2). Threading outcomes throughout learning is seen as important for the relevance of learning experiences. Schroeder (1993) argues that learners prefer learning in context; they prefer practical and concrete experiences. An outcomes approach is seen as being an impetus for such learning because it suggests the need for contextual types of activities and assessments (Mentkowski et al., 1991). Learning outcomes are also described as beneficial from an instrumental perspective. The language of "competency" and "skill" is often used to articulate the needs of business and industry, and the imperative for education to lead economic recovery (Meadmore, 1995). When combined with the historic link of competencies and trades education, it is difficult for some educators to acknowledge the existence of competencies reflecting complex cognitive skills. Dunne et al. (1997) discuss the language shifts that have occurred in the United Kingdom because of the range of interpretation and the connotations associated with terms. The concepts of core skills and transferable skills have been shaped into the idea of key skills. Language can present a barrier to communication, particularly when working towards integrated learning involving educators from several disciplines (Wilde & Hardaker, 1997). The focus on abilities and learning outcomes could serve an instrumental function to promote communication among educators from diverse program areas and organizational contexts. An outcomes based approach is viewed as an impetus to shift the purpose of educational institutions from a focus on teaching to a focus on learning (McDaniel et al., 2000). It challenges the foundations of our educational institutions that have traditionally been focused on educators (Hutmacher, 1997). This is perceived to make faculty more "learner-centered" (Davis & Felknor, 1994).  47  The notion of learning outcomes has shifted the focus in education circles from teaching to learning. Implicit in the discussion of a 'learning system' based on explicit statements of outcomes is the belief that decisions concerning curriculum design, instructional design, content and delivery are based on assisting students to achieve the [desired] outcomes. (Stanley & Mason, 1997, p. 5) Such an approach encourages consensus building around collective outcomes (McDaniel et . al., 2000). Faculty members relate their courses more directly to collective abilities articulated by their departments, organizations and / or national policy frameworks. Mentkowski (1983) presents a similar perspective; she argues that an abilities-based approach provides faculty with a basis for a vision that can bring about organizational cohesion and change. Learning outcomes models can also be designed to support prior learning assessment and recognition initiatives. They can help to reduce traditional entry barriers (Burrow, 1993; Ecclestone, 1994; Matthews, 1997; Simosko, 1997). From this perspective learning outcomes facilitate credentialing processes and also support the demonstration of accountability (Wilde & Hardaker, 1997; McDaniel et al., 2000). Issues of accountability are threaded throughout the discussions about the benefits of learning outcomes. Linking assessment to learning provides evidence about the learning within higher education as discussed by Mentkowski et al. (1991). The consensus building process around the articulation of learning outcomes also supports accountability of individual educators. Transparency becomes evident when "faculty members no longer teach and test behind closed doors" (McDaniel et al., 2000, p. 146). The accountability discussion can thus be framed within the context of abilities and their assessment. Overall the literature suggests many positive aspects about a learning outcomes approach from pedagogical as well as instrumental perspectives. Perceived Limitations The idea of assessing the outcomes of learning seems reasonable and logical. Why is it that this idea has been the focus of such debate and controversy? Manno (1994) suggests that the "devil is in the details." In this section I explore the details in the literature and review the controversies surrounding the concept of a learning outcomes approach. The most aggressive critiques are found in the literature from the United Kingdom and reflect the debates surrounding the system of national vocational qualifications. However, similar themes are found in the New Zealand and Australian literature. While these  48  discussions revolve around the idea of competencies, the outcomes language blends in the world of practice and this literature is, therefore, relevant to my study. While the proponents of outcomes approaches see it as a vehicle for bridging the gap between liberal and vocational education (Holland, 1993; Carmichael, 1993; Bauslaugh, 1997b), the opponents view this as a strategy shifting the purpose of education towards an economic focus. "In all sectors, including higher education, 'bridging the gaps' between 'general,' 'vocational' and 'academic' education and training is now assumed to have one purpose: increasing learners' employability" (Ecclestone, 1994, p. 159). There has been a subtle shift in language that reflects more profound changes in higher education (Hutmacher, 1997). Ecclestone argues that words, which have traditionally been associated with liberal education such as personal development and life-long learning, are now being framed in corporate contexts. She suggests that discussions about the aims of higher education have been subsumed by economic aims to the point that discussions about democratic values embedded in broader social and political contexts are criticized for being elitist, exclusive and irrelevant thus silencing debates about their importance. Academic competence is being displaced by the notion of technical competence (Ecclestone, 1994; Dunne et al., 1997). Rather than bridging the gap, the new vocational focus is narrowing the focus of higher education (Soucek, 1993). The debate revolves around whose outcomes are going to count. Avis (2000) contends that policy makers assume that the needs of the economy and learners are similar. This leads to the domination of education by market perspectives that may not necessarily be valid (Axelrod, 2002). Joyner (1995) and Avis suggest that an outcomes approach may become a form of empiricism reflecting current perceived employer needs that may not necessarily address the abilities required by today's economy. Curtain and Hayton (1993) present a similar concern about the Australian standards framework, which "discourages labour flexibility and creates a career pathway based on outdated hierarchical concepts of work organization" (p. 15). The market relationship of learning outcomes is a frequently voiced concern (Jackson, 1993; Ecclestone, 1994; Whitston, 1998). One argument is that a structure that has theoretical roots in the work place may be limited in scope, and may not be able to address the emergent skills needed to survive in a knowledge-based economy (Canning, 1998). The employment focus may represent a narrow economic perspective, and shifts attention away from the wider social and political contexts that are critical to democracies. Concern is expressed that current  49  approaches to outcomes based education may "irrevocably narrow education's wider social values and purposes" (Ecclestone, 1994, p. 163). Jarvis (2000) expresses a similar concern about the scope of learning. He identifies the risk that only learning that "is recognized by some form of award becomes defined as 'real' learning, while all the other human learning that helps make people what they are will be neglected and regarded as unreal - and even unnecessary - and lifelong learning will become equated with worklife learning" (p. 63). Ainley (1998) raises a similar issue. He suggests that policy directions are focused on credentialism not on learning. We are creating a certified society, not a learning society. There are concerns that outcomes based education will normalize the educational experience to the lowest common denominator, and produce a mechanistic view of learning that oversimplifies the complex process learning involves (Collins, 1993; Soucek, 1993; Hutmacher, 1997; McDaniel et al., 2000; Avis, 2000). In particular, questions arise regarding the prominence of knowledge within the outcomes frameworks (Ackerman, 1998; Canning, 1988). While the Alverno model clearly articulates the perspective that knowledge and understandings underpin performance (Loacker & Mentkowski, 1994), the fundamental position of knowledge is not as clearly evident in the other approaches (Soucek, 1993; Hutmacher, 1997; Canning, 1998). These concerns about a rigorous theoretical and conceptual base are sometimes expressed through the notion of "dumbing down" the curriculum (Davis & Felknor, 1994). The notion of transferability of skills is often described as a feature of outcomes based education (Borthwick, 1993; Shipley, 1995; Wilde & Hardaker, 1997). This notion of transferability is based on the assumption that acquisition of abilities in one area has the potential to be applied in other areas and contexts (Allan, 1996). However, Ackerman (1998) argues for the importance of knowledge in both learning and performance. Learners' success at intellectual tasks are influenced more by their knowledge than by their ability in abstract reasoning. This position is supported by others (Gagne, 1977; Stanley, 1993; Balin, Case, Coombs, & Daniels, 1999). Developing expertise is domain specific; it is a process by which individuals adapt to the specialized styles of learning and thinking associated with a domain and become increasingly different from novices in the area but also experts in other areas (Stanley, 1993). Experts have a vast knowledge that supports their abilities; a set of strategies is necessary but not sufficient for critical thinking (Balin et al., 1999). The ability to transfer approaches from one domain to another is dependent on the knowledge base possessed by  50  learners. Relationships among abilities, teaching approaches and knowledge domains are complex. Stanley suggests that we have not reached the stage in our understanding of learning to suggest that one model should be considered the panacea for our educational approach. The relationship between performance and competence is another area of contention (Hutmacher, 1997; Soucek, 1993; Stanley, 1993). The proponents of an outcomes approach underestimate the complex relationship between knowledge and action. Central to this issue is the distinction between performance and competence. Competence is inferred from performance and in the cognitive domain this can be problematic. "Observation of performance cannot ignore the issue of meaning, especially the meaning given to situations by subjects, the implications they see in them and the interpretations they give to them" (Hutmacher, 1997, p. 46). One has to be careful in making such inferences, and the inferences become less reliable when moving from simple to more complex tasks (Stanley, 1993). Performance is not a guarantee of underpinning knowledge (Soucek, 1993). Competence goes beyond performance to knowledge and understanding underpinning the actions. Competence rests on "an integrated deep structure (understanding) and on the general ability to coordinate appropriate internal cognitive, affective and other resources necessary for successful application" (Wood & Power, 1987, p. 414). Ryan (1998) argues that the competency movement creates a means - end divide that does not incorporate the complexity of teaching and learning. It undermines the values and knowledge that are integral to education, but may not be explicitly evident atfirstglance. Strathern (2000) expresses a similar concern in her article entitled The Tyranny of Transparency. By emphasizing one type of reality, other perspectives are eclipsed. By focusing on immediate assimilation of information, the long-term effects of the learning experiences may be missed. These long-term effects may occur weeks or years after the event, and may present in forms that do not resemble the original focus. Another example relates to the focus on implementation. Hutmacher (1997) argues that an overemphasis on abilities may result in the neglect of important questions surrounding the acquisition of abilities. By focusing on the product we underestimate the complexity of learning and variables that contribute to learning. We "need to recognize how models of teaching and learning are never innocent; that they derive from particular socio-economic contexts and construct teachers and learners in particular ways" (p. 12). Avis (2000) identifies the same concern about the perceived innocence of approaches to learning.  51  Nordhaug (1997) argues that education is too complex and diverse for the application of one approach. He argues that we need to determine the approaches that would be most suitable for different types of education. Canning (1998) would agree with this approach in that he suggests that learning outcomes may be more relevant in applied areas. These perspectives highlight concerns regarding the use of a learning outcomes approach, and its instrumental and simplistic approach to education. While the qualification standards in the United Kingdom, New Zealand, and Australia were designed to integrate vocational and liberal education, in practice they predominantly focus on narrowly defined tasks (Curtain & Hayton, 1993). This resulted in concerns about the "ghettoisation" of the qualifications in the United Kingdom (Hodgson et al., 2001). They "prepare people for a life of dependence and powerlessness. Far from contributing to the strengthening of the democratic values of fairness, justice and equality, they deny them and replace them with benign exploitation, servitude and coercion" (Hutchinson, 2000, p. 91). Avis (2000) raises a similar concern. "The paradox is that we confront a language of empowerment that operates on a terrain that encourages student passivity with this process being compounded by managerial needs for control and information" (Avis, 2000, p. 10). Ainley (1998) expresses a similar concern through the idea of skill-polarization. The rhetoric of empowerment is not evident in practice. In the UK and Australia concerns are expressed about outcomes being too narrow, detailed and prescriptive (Collins, 1993; Curtain & Hayton, 1993; Avis, 2000). They are easy to monitor, observe and audit, but they may curtail innovation, flexibility and reduce learners' autonomy to define their own outcomes (Soucek, 1993; Ecclestone, 1994). In the American context, they are described at nebulous and hard to measure (Manno, 1994; Drake, 1997). In discussing the challenges of assessment Bruneau and Savage (2002) address this conundrum. "The more precise benchmarks become, the more they resemble a legislated national curriculum, with all that implies for political orthodoxy; the less precise they are, the more they look like academic hot air" (p. 97). Drake (1997) addresses a further ambiguity. She contends that many of the "transformational" outcomes (Spady, 1994) are value laden and challenging to measure. She concludes that this reality "makes the intended outcomes just so many empty words rather than a focus for learning" (p. 45). Outcomes that are too narrow are restrictive and limiting; outcomes that are too broad become meaningless. The idea of "writerly" and "readerly" texts (Barthes, 1970; Hawkes, 1977) may help to explain some of the critiques made of a learning outcomes approach. Barthes suggests that  52  some texts provide the reader a role, an opportunity to make a contribution (readerly texts). Other texts only leave the reader with the option to reject or accept the text (writerly texts). The detailed and specific texts related to learning outcomes from the UK, Australia and New Zealand could be viewed as writerly texts. There are no opportunities for learners, to shape and to contribute to their development. This may account for the ghettoisation, skillpolarization and related critiques made by Hodgson et al. (2001), Avis (2000) and Ainley (1998). The writerly texts facilitate auditing but do not empower learners. Many of the texts from the United States can be viewed as readerly texts. They lend themselves to interpretation and shaping. Such texts may be more palatable to educators, but may not meet the needs of policy makers interested in quantifiable outcomes (Manno, 1994). Despite his strong critique of outcomes approaches Avis (2000) also identifies the possibility of a readerly approach; his concern is that a readerly approach is only one of two directions in which the learning outcomes approach may be shaped. Jackson (1993) and Avis (2000) suggest that the outcomes approach has more to do with managerial control and reporting than with learning. Jackson argues that the competency approach is an ideological practice through which governance takes place. Outcomes approaches provide a "conceptual framework, a vocabulary and set of institutional practices through which local educational activities are subordinated to the dominant political discourse of our time" (p. 156). It promises to provide more clearly defined goals and measures to assess the outcomes of public funds in higher education. Such approaches represent "new forms of bureaucratic surveillance" that also undermine professional autonomy (Ecclestone, 1997). Bruneau and Savage (2002) raise a similar point in their discussion about performance indicators. They "have next to nothing to do with liberal education, but everything to do with market discipline and control" (p. 217). Learning outcomes are being offered as "a placebo for a coherent economic and industrial strategy" (Jackson, 1993, p. 159); they represent one of many forms of performance indicators. While the BC context may differ, one can see the potential application and implications of an outcomes approach and its potential link to performance funding. The literature on learning outcomes provides a spectrum of views. These views are grounded in the political and educational context of the authors' work and lives. The plethora of terms related to outcomes of learning, and the shaping of these concepts provide opportunities for policy makers and educators to influence their direction. Mentkowski  53  (1998) suggests abilities (read learning outcomes) could form the cornerstone for implementing an organizational vision. Conversely they could result in the creation of a marginalized "underclass" through the lack of worthwhile credentials (Ainley, 1998). Given . these two potential directions, it is no wonder that the notion of learning outcomes evokes such passion in educators. In the next section, I will go beyond the proponents' and opponents' views and delve into the literature that supports the importance of a learning outcomes approach. Relationship Between Learning Outcomes and Educational Reform Although the learning outcome approach is supported through testimonials, there is little research in post-secondary education to substantiate its proposed central position in educational reform. Much of the literature is directed towards outcome assessments in general, but little information is available regarding the value of a learning outcomes approach. This emphasis in the literature may be a result of the pressure exerted by national and state governments (SCANS, 1995; Berman, 1995) and by American regional and specialized accreditation organizations in the late 1980s (Derlin et al., 1986; Manno, 1994; Salvador, 1996). In this section I review the research pertaining to learning outcomes and also . explore the research behind and beyond outcomes as it pertains to what we know about learning in postsecondary education.  Research related to learning outcomes. In the United States the requirement for outcomes assessment by accreditation organizations appears to have influenced research in this area. Much of the literature and the studies in postsecondary education are directed towards the evaluation of specific learning outcomes. For example, researchers investigate the evidence to support the claim that graduates have demonstrated identified exit abilities and knowledge. The studies are primarily based on standardized test results (Howard Community College, 1991; Evans & King, 1994; O'Neil, 1994; Marzano, 1994; Berman, 1995). Graduate scores on national examinations are tracked to analyze their relationship to generic learning outcomes. For professional programs, licensing examination results are used to evaluate graduates' knowledge and abilities. Other evidence used to support an outcomes approach includes data related to student satisfaction, student transfer, student retention, and employer satisfaction (Howard Community College, 1991; Smith, 1992; West, 1994; Berman, 1995). These studies suggests that students and employers are satisfied with the educational outcomes of programs. This  54  outcomes evidence is, however, not necessarily related to programs using an outcomes based curriculum approach at the program and course level as suggested in the BC context. As such it does not provide substantive evidence to support a provincial shift towards such a framework. Alvemo College has integrated an abilities approach for over twenty years and has been a major national force in promoting this approach through publications and faculty development workshops. The faculty and administrators at Alvemo believe in the need for longitudinal studies to assess the development of their students' abilities (Mentkowski et al., 1991), and have allocated resources to this end. The following information provides some examples of their research in this area. Hart, Rickards and Mentkowski (1995) conducted a longitudinal study of Alvemo students from the 1976 and 1978 freshman class investigating the intellectual and ethical development of learners' abilities during their programs and their career paths upon graduation. The researchers used a variety of theoretical frameworks to measure this development. They found development over a ten-year period in three areas, classroom learning, decision-making and career decision-making. This development applied equally to those who entered directly from secondary school, those who transferred to the College, those who delayed their post-secondary education for a period, and those who had previous careers, although there were some differences among these groups. This study supports the view that learners demonstrate development during their college education and continue to develop after graduation. Mentkowski, Much and Giencke-Holl (1984) conducted a study of 60 Alvemo graduates two years after graduation through a survey instrument with follow-up interviews. They found that interpersonal abilities and reasoning abilities were important in the graduates' work experiences. Based on their analysis of the results, the researchers suggest that "learning to learn" is the link between education and work. Educational abilities provide the foundation, but the ability for independent learning is required to adapt the abilities to new experiences. This may be an important feature of an abilities-based approach. Alumni were found to use the abilities from their education to create an approach to action that was tested and validated in their work environment. Ben-Ur and Rogers (1994) conducted a study to measure the career advancement of 5-year graduates from Alvemo College. This involved a sample of 243 graduates who were predominantlyfirstgeneration college students from working class backgrounds. Participants  55  were asked to rank their current positions with regard to abilities that were required in their work. The researchers also assessed the participants salary scale and conducted an interview during which the participants' abilities were explored through a position autonomy scale including "discretion or authority granted, ... level of expressions (oral or written) required, ... initiative and original thinking required, and ... level of judgment required" (p. 8). This study was based on the assumption that education abilities were translated into career abilities. This raises the obvious question regarding workplace experiences and their relationship to the communication and thinking abilities studied. However, the study does support the idea that abilities are important in the workplace and that they can contribute to career development. Mentkowski (1990) identifies several challenges associated with the measurement of change from an abilities-based approach. Abilities-based education requires assessing complex outcomes for which the unit of measurement is often not a test item, rather the learners' performance. Such a performance requires a qualitative approach based on expert judgments. Further challenges arise if we accept the assumption that change is non-linear as supported by Pascarella and Terenzini (1991). Other variables such as maturation need to be factored into the analysis as well. These challenges account for the longitudinal approach adopted by Alverno in assessing the outcomes of their programs. However, such an approach may not meet the needs of policy makers who are looking for more immediate indicators (Webber & Townsend, 1998). Loacker and Mentkowski (personal communication, June 1998) addressed the 6  challenges of comparative analysis. Alverno students take many standardized tests during their education but these tests do not necessarily test the abilities that are the basis of the Alverno program. These tests indicate that Alverno students and graduates have comparable scores to those of other colleges and universities. The graduates also have similar patterns of acceptance into graduate programs at other universities when compared to graduates of other organizations even though the Alverno transcripts involve descriptive data rather than quantitative data. Based on the research conducted by Mentkowski and her colleagues it appears that graduates of Alverno succeed in the workplace and they succeed in postgraduate work. While the previously described Alverno studies (Mentkowski et al., 1984; Ben-Ur & Rogers, 1994; Hart et al., 1995) suggest that abilities-based education supports learner These discussions occurred during Alverno workshops on learning and assessment.  56  development, it is challenging to determine the superiority of abilities-based education in general when compared to other types of curriculum approaches. There may be such a difference but it is difficult to capture through quantitative approaches. Qualitative approaches have other challenges, particularly in the area of comparative conclusions. Yet it is precisely such comparative data that many educators seek in looking at the issue of curriculum change. A study was conducted in England by Dunne et al. (1997) to identify ways of developing cores skills in higher education. They studied the practice of 32 university educators in 16 departments including a mix of vocational and non-vocational departments. Student perspectives (n = 350) were gained to further study the practices of these educators. Despite the context of core skills in these departments, the researchers found that the educators tended to articulate their goals in more disciplines oriented language, and the students as well focused on discipline oriented outcomes when describing the value of their educational experiences. "In response to a question on 'skills learned from the module,' 52% wrote about content and subject knowledge, not about skills" (p. 516). When looking for the teaching of such skills, the skills were often not evident at the implementation and assessment phase even though they were theoretically incorporated. The researchers argue that the rhetoric related to the development of core skills is not espoused by either the educators or the students. However, 43% of the students described the benefits of a degree in terms of employment opportunities. They suggest that the concept of core skills is an example of an "unfulfilled concept," one that "is not sufficiently coherent in the abstract to be fully 'realized' in practice" (Stalker, 1996, p. 12). The support for a learning outcomes approach is mainly derived from a deductive reasoning process. However, educators question the inductive evidence to support the integration of a learning outcomes approach. In particular they question the priority given a learning outcomes approach and its potential implications for higher education. "The whole current discourse is dangerous because it shifts the balance of power in the wrong direction and threatens crucial educational purposes in a democratic society" (Collins, 1993, p. 11). It "may ... irrevocably narrow education's wider social values and purposes" (Ecclestone, 1994, p. 163). On the one hand a subtle change in language is perceived as having the potential to be the impetus for fundamental change (Ecclestone, 1994; Hutmacher, 1997), while at the same time changing the vocabulary related to an outcomes approach does not change the conceptual and practical challenges associated with this approach (Dunne et al.,  57  1997). These challenges lead to questions about the relative importance of curricular organization to student development. The debates surrounding this issue form the basis for the next section.  Behind and beyond learning outcomes.  While the importance of the curriculum is  largely taken as self-evident, the organization of the curriculum based on outcomes approaches is questioned (Stanley, 1993; Hutmacher, 1997). In fact Collins (1993) suggests that this learning outcomes orientation was essentially a phenomenon in English speaking countries being mainly adopted in the vocational and professional areas. However, the discourse has now been extended to general education sectors as well in Australia, the United States and Canada. The emphasis on learning outcomes as a model for all areas of higher education is the issue being questioned. Hutchings (1999) contends that the current focus on outcomes is limiting when it comes to making improvements in the learning environment. She suggests that we need to get behind outcomes and broaden our approach to assessment. Researchers such as Mentkowski, Astin, Ewell, and Moran suggest that we need to place more emphasis on assessment as part of learning and question the traditional assumptions we have made about assessment (Mentkowski et al., 1991). These researchers suggest that we need to gain a broader view of learners, learning, and educational experiences if we are to effect positive change in higher education. The importance of the curriculum on learning is questioned (Astin, 1993; Pascarella & Terenzini, 1991; Garcia-Diez, 2000). For example, a reform initiative was introduced in Spanish colleges that focused on an increased emphasis on practical content within courses. In her study offirstyear economics students Garcia-Diez measured cognitive effect (in terms of test scores) and affective effect (in terms of interest) of the new curriculum during its first year of implementation. While Garcia-Diez identifies the limitations of studying the new curriculum during itsfirstimplementation year, no significant differences were found in terms of student cognitive achievement; however, there was an effect on learner interest with an increased level of interest with the new, more practical curriculum. It could be argued that the test items may not have captured the learning from a practical perspective. In her analysis of two studies from Australia, Brady (1999) found that the introduction of a learning outcomes approach did not significantly affect classroom pedagogy. Educators were making  58  their learning explicit, but there seemed to be little change in the classroom as evidenced by data from interviewees and observations. Hutmacher (1997) addresses the issue of feasibility when discussing the implementation of abilities-based models. He argues for a systemic approach that includes implementation and assessments issues as well as organizational culture. There are a plethora of variables that have an impact on learning and the focus on outcomes of learning creates a means-ends divide that fails to acknowledge the complexity of teaching and learning (Ryan, 1998). By overemphasizing the debate about abilities, questions of implementation may be neglected. Articulating curricula is not enough. The acquisition of abilities requires that learners be actively involved (Hutmacher, 1997). The importance of other variables is supported in the literature. For example, Astin (1993) found that variables associated with general education curriculum had a weak influence on student development, whereas peer involvement, faculty interaction and participation in out-of-class activities had greater influence on student development. Pascarella and Terenzini (1991) found similar results with respect to peer, faculty, and extracurricular involvement. Garcia-Diez (2000) in her study offirstyear economic students in a Spanish colleges also found peer group involvement to be important for learner achievement. Social involvement appears to be an important factor in student development. Astin (1993) argues that students' peer groups are "the single most potent source of influence on growth and development during the undergraduate years" (p. 398). He also identifies challenges for colleges in this regard. The combination of the diversity of the student population (particularly with regard to age) and the absence of a residential experience makes it difficult for students to identify with each other and establish supportive relationships. Donaldson (1999) expands on this theme. He contends that adult learners with their complex lives and commitments may use instructional time differently than the more traditional students. They may use this time for enhancing their interactions with peer and faculty members, because they do not have the time to become involved in out-of-class activities. Hutmacher (1997) also contends that educators' personal commitments are important to learning. Grubb (1999) in his research found that the quality of teaching in colleges depended on the individual educator. Entwistle (2000) supports this position. He argues that assessment strategies have a pervasive influence on learning. He contends that teaching and assessment strategies affect the learners' balance between deep and surface learning, and thus  59  influence the learning outcomes achieved. The organization of learning experiences is an important variable when assessing student development. Chickering and Gamson (1991) have also been instrumental in identifying principles of good practice in undergraduate education. Their principles include: encouraging contact between students and faculty, encouraging cooperation among students, encouraging active learning, providing prompt feedback, emphasizing time on task, communicating high expectations, and respecting diversity in learners including learning styles. Sorcinelli (1991) suggests that these principles provide ways of enriching our understanding of teaching and learning, but we still need to explore the relationship between the principles and other variables such as student motivation and development in more depth. These highlights from the literature on teaching and assessment serve to emphasize the multiplicity of variables that influence the outcomes of higher education. They provide a context for understanding the questions raised by opponents of learning outcomes. As Whitston (1998) so aptly stated, "key skills [learning outcomes] may be a poor substitute for more radical curriculum reform" (p. 308); learning outcomes may also be a poor substitute for educational reform in general. There is an inherent risk associated with the emphasis on outcomes. Learning outcomes place so much emphasis on the product of education that the process of learning may be marginalized. Given the climate of fiscal restraint, there is a potential risk that the needs of learners may be subsumed by financial considerations. The learning outcomes initiative in British Columbia was a policy direction that warranted further investigation. To further the exploration of this learning outcomes policy I review the concept of policy and describe a policy framework that will support my analysis.  The Policy Process Defined There exist a plethora of definitions for the conception of policy; they focus both on process and product elements (Howlett & Ramesh, 1995; Parsons, 1995; Turner, 1997). Wildavsky (1997) goes so far as to question whether the policy making process can even be defined. Given this complexity, I will use a broad definition of policy, one that focuses on the i