UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Redefining accountability in adult basic education : an emancipatory approach to student assessment in… Moss, Raymond Douglas 2007

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


831-ubc_2007-318935.pdf [ 10.32MB ]
JSON: 831-1.0100719.json
JSON-LD: 831-1.0100719-ld.json
RDF/XML (Pretty): 831-1.0100719-rdf.xml
RDF/JSON: 831-1.0100719-rdf.json
Turtle: 831-1.0100719-turtle.txt
N-Triples: 831-1.0100719-rdf-ntriples.txt
Original Record: 831-1.0100719-source.json
Full Text

Full Text

REDEFINING ACCOUNTABILITY IN ADULT BASIC EDUCATION: AN EMANCIPATORY APPROACH TO STUDENT ASSESSMENT IN BRITISH COLUMBIA'S SCHOOL DISTRICTS by RAYMOND DOUGLAS MOSS B.G.S., Simon Fraser University, 1986 M.Ed., University of British Columbia, 1993 A THESIS IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF EDUCATION in THE FACULTY OF GRADUATE STUDIES (Educational Leadership and Policy) THE UNIVERSITY OF BRITISH COLUMBIA August 2007 © Raymond Douglas Moss, 2007 Abstract This retrospective case study chronicles and critically interprets the design, development, and eventual demise of a student-centred mechanism for assessing accountability in adult basic education (ABE). ABE is understood as an attempt to address the learning needs of adults at risk of economic and social marginalization and provide them opportunities to further learning. From 1998 to 2001 the British Columbia's Ministry of Education was engaged in examining the accountability practices of A B E programs offered by the province's school districts. In the final two years of that timeframe it developed in collaboration with the field an accountability framework intended to be capable of acknowledging the many ways program success might be demonstrated and measured; adaptable to demographic, economic, and cultural changes; sensitive to local differences, learner needs, and political priorities; and emancipatory in its orientation toward learners and program providers. Implementation of the framework was stayed due to a change in government. This study reflects upon my experiences as the school district senior administrator in adult education seconded by the ministry to be the framework's principle architect. Through examining the idiosyncratic political and professional context of developing ABE accountability policy for British Columbia's school districts, this study uncovers more universal themes associated with the power dynamics of teaching professionals, their various governmentalities, and the social construction of failure and success. This lays the basis for further studies in the political economy of accountability policies and practices for adult basic education. Table of Contents Abstract ii Table of Contents iii List of Tables vi List of Figures vii Glossary viii Acknowledgements... xii Dedication xiv Chapter One 1 Background 1 The Purpose of this Study 7 The Research Question 8 My Relationship to the Study 8 Approach to the Study 10 Sources of Evidence 11 Reflection, Organization, and Interpretation 15 Preconceived Ideas Brought to the Study 16 Delimitations and Limitations 19 The Study's Significance 19 Structure of the Thesis 20 Chapter Two 25 Background 25 System Differences , 29 Planning an A B E Quality Framework 35 The Question of Student Success Rates 39 Toward a Coordinated System 44 In Pursuit of a Common Accountability Framework 46 Summary 49 iii Chapter Three 52 Adult Education 52 ABE Students.... 55 Educational Accountability 61 Economic Accountability 63 Social Accountability 65 Technical Aspects of Accountability 67 Sociopolitical Implications 69 Designing Accountability Frameworks 73 Summary 84 Chapter Four 87 A Conflicted System 87 Multiple Urgencies 87 Partial Perspectives 90 Fragmentary Ideals 92 The Provincial School District Review 94 Constructing the Framework 99 Informing Elements 102 Fiscal Accountability 102 Federal Funding Allocations to First Nations ABE Programs 104 Enrolment 105 Attendance and Progress 107 Admissions 109 Form, Function, and Purpose 110 Summary 116 Chapter Five 131 Reviewing the Framework 133 Program Assessment 135 Student Assessment 136 Toward Emancipatory Accountability 137 iv Professionalization 138 Governmentality 139 Voice and Power 141 Core Course Development 141 Student Diversity 142 Nontraditional Indicators of Student Success 144 Devising Alternative Pathways to Graduation 148 Software Coding of Qualitative Indicators 148 The Review Process 149 Summary 154 Chapter Six 158 My Secondment: Leadership in Context 158 The Policymaking Process 160 Constructing the Framework 162 The School District Review 173 Political Realities 175 The Current Situation 177 Summary 182 Chapter Seven 186 Overview ' 86 Implications for Educational Policy and Practice : 194 Suggestions for Further Research 197 References • 200 Appendix 1 205 Appendix 2 221 Appendix 3 223 Appendix 4 224 v List of Tables Table 4.1: Goals and Objectives of Adult Basic Education 92 Table 5.1: Nonacademic Learning Outcomes Contextualized to One Learning Environment ...145 vi List of Figures Figure 1: The Accountability Framework 120 vii Glossary of Terms To assist the reader in understanding this study, the following definitions denote the manner in which I use certain terms, or indicate how the terms are publicly defined. Academic Success Indicators: Traditional indicators of school and student success that include standardized testing, course completions, and graduation rates. Adult Basic Education (ABE): For the purpose of this report A B E refers to K-12 courses in which adults enroll to complete requirements for grade twelve graduation or, in the case of high-school graduates participating in ABE, to upgrade their qualifications and gain the necessary prerequisites for employment or further education opportunities. Adult Students: The school act refers to adult students as 19 years of age and over as of July 1st for the school year commencing that September. British Columbia Adult Graduation Certificate (Adult Dogwood): For students 18 and older, the certificate is comprised of five courses at the grade 11 or 12 level, and is the common graduation certificate for adult students in school districts and colleges. British Columbia Regular Graduation Certificate (Regular Dogwood): For school-aged students and adult students, the certificate is comprised of 52 credits at the grade 11 or 12 level. Although the 52 credits can be made up of two and four credit courses, the majority of academic courses are four credit courses. British Columbia School District Continuing Education Directors' Association (BCSDCEDA or CEDA): This association of school district continuing education administrators meets four times a year to discuss issues concerning their respective program areas. The association is in continual contact with the MoE regarding A B E policy and a ministry representative attends CEDA meetings. Continuing Education: Within the context of this study continuing education refers to any part-time, short-term, or long-term educational program or activity that may be taken for academic or general interest. Continuing Education (CE) Number: For the purpose of this study a CE number refers to a school code designated by the Ministry of Education. Students enrolled in a CE centre must be 16 or over to qualify for ministry funding and the district is required to report fundable students in September and May of each school year. School districts are funded on the larger count. Adult students and school-aged students are funded differently. Full Time Equivalent (FTE) - Adult: Refers to a full-time equivalent student for the purpose of funding. A school district receives $4,000 per year for each full-time adult student. From 1998 to 2001 ABE students were funded on the basis of the number of courses they were registered in, with eight courses required of a full-time student. A school district received 1.25% of the funding for each course the adult student was registered in. As of 2004 funding policy is no longer based on the number of registered students but on course completions. Full-Time Equivalent (FTE) - School Aged: Refers to full-time equivalent school-aged students for the purpose of funding. A school district receives $5,300 per year for a full-time school-aged student. Students are funded on the basis o f the number o f courses they are registered in. Ministry of Education (MoE): The M o E is the Bri t ish Co lumbia ministry responsible for the K-12 education system, including school district A B E programs. Ministry of Advanced Education, Training, and Technology (MAETT): M A E T T was the name used by the B C ministry responsible for the postsecondary education system during the timeframe o f this study. Its responsibilities include A B E programs offered through colleges. Non-Academic Success Indicators: These indicators o f school and student success include social ski l ls , self-confidence, meeting deadlines, asking for help, patience, working with others, initiative, and determination. Traditional Learning Environments: Structured classroom learning environments, typical o f most high schools. Teachers usually teach one or more subjects that are illustrated on student timetables as blocks. Teachers generally remain in their areas o f expertise, teaching several blocks to various grade levels or classes. The courses are taught at regularly scheduled intervals for pre-specified periods o f time. Traditional learning environments are frequently associated with large institutions such as high schools which often enroll more than 1,000 students. Nontraditional Learning Environments: These learning environments are often established by school district adult/continuing education departments in an attempt to provide an opportunity for adult students to resume their education with as little difficulty as possible. The physical environment is often a storefront operation containing some small workrooms and larger open-area workspaces. It usually enrolls a limited number of students, generally from 75 to 250, and programs are frequently offered on the basis of continuous entry. Students work at their own pace, and are not assigned a particular work area, but choose their own. As the students work through the coursework, which is generally modular in design, teachers circulate through the work areas, providing assistance as needed. Teachers who are specialists in the design, implementation, and teaching of numerous academic courses supervise the courses. The uniqueness and success of the program demands that the teacher's expertise not be limited to one subject area. School Aged Students: The school act refers to school aged students as those who have not reached the age of 19 as of July 1st for the school year commencing that September. 1701 Counts: School District A B E Student Registration Forms, referred to as "1701s", are counted in May and September as one means of determining the number of students registered. xi Acknowledgements I am indebted to many people for their contributions to this study. First, those at the British Columbia Ministry of Education who at the time of this study recognized the complicated nature of adult basic education and supported me in my endeavor to develop an A B E accountability framework. Without their support this research would not have been possible. The guidance of Dr. Kjell Rubensen, Dr. Tom Sork and Dr. Garnet Grosjean as my committee members has been profoundly helpful. Their patience and encouragement helped to keep me on track through a long and mostly difficult journey of reflective exploration as I unpacked this introspective story of self-discovery. I am very grateful for their support. I am thankful as well to Dr. Carolyn Shields who worked with me on this project until accepting a new position at the University of Illinois. Much appreciation goes to University Examiners Dr. Pierre Walter of the Department of Educational Studies and Dr. Ken Reeder of the Department of Language and Literacy Education, both of whom asked challenging, insightful, and edgy questions that nevertheless were a joy to engage. Examination Chair Dr. Ann Anderson of the Department of Curriculum Studies added decorum and humour to an otherwise stressful event. Hearty appreciation also goes to External Examiner Dr. Nicholas Rubidge, President of the College of the Rockies, whose critical questions and profound historical knowledge of ABE in British Columbia contributed to the veracity of the final version of the dissertation. A number of friends read drafts of the thesis as it moved through various stages of completion. Though leading very hectic lives these individuals took time to review the thesis and provide xii valuable feedback. Louise Thorburn and Dawn Minty helped move the project closer to completion. I would like to thank the many professionals, both in the ministry and the field, who shared their time and expertise to help shape this study. I am particularly grateful to those anonymous professionals who conducted the informal review of the A B E framework; they gave unselfishly of their time, energy, and expertise, and their efforts contributed significantly to my first-hand understanding of the emancipatory possibilities of framework. My friend and colleague Kevan Brewer contributed to the design and ease of the PowerPoint presentation of my oral defence for which I most obliged. I am also thankful and indebted to my editor and friend Dawn House whose support and expertise was invaluable in assisting me to complete this dissertation. Finally, I would like to thank my parents Ray and Renee Moss for their love and support through life and this project. X l l 1 Dedication To my mother and father, Renee and Ray Moss, whose knowledge, moral understanding, and wisdom provided me with a sense of self. xiv Chapter One Introduction Background Publicly funded education programs have come under increasing scrutiny as governments have responded to taxpayer concerns through politicized accountability initiatives that all too often fall victim to economic agendas. Accountability structures have become an integral aspect of publicly funded education programs that are increasingly shaped by "world-changing economic, social and political processes" (Lipman, 2006, p. 52). Education is often discussed as a practice through which men and women are formed and reformed to become social individuals (ibid.). Governments often design politicized accountability initiatives that are aligned with economic agendas. When applied to education the potency of economically motivated accountability structures drive the development and outcomes of educational accountability frameworks. In the current political climate, policies emphasizing an economic approach to educational accountability tend to reflect a neoliberal agenda which is aligned with a corporate mentality and focused on system outcomes. Standardized tests and course completion rates are the preferred measures of student performance and educational efficiency and effectiveness. An over-reliance on these measures is questionable in most K-12 educational formats as it diverts attention from other purposes and goals of education such as citizenship, social skills, and employment skills (Ben Jaafar and Anderson, in press). However, it is a particular disservice to learners engaged in adult basic education (ABE) programs,1 many of whom were marginalized by performance indicators that focused on system outcomes while failing to value factors such as adequacy of resources, school capacity, regional cultures, and the social and familial milieu impacting student motivation. A more appropriate framework for understanding educational accountability in an ABE environment would assume a more educational alignment that focused on the needs of the student and utilized success indicators that reflected student performance in terms of personal and social growth and development in addition to test scores and course completion rates. This study chronicles and critically interprets the design, development, and eventual demise of such a student-centred accountability initiative from the perspective of one of its architects. In presenting this story the thesis explores the complexities involved in constructing and implementing an accountability framework for publicly offered school district ABE programs in British Columbia. As the tale unfolds, the educational dynamic of A B E in British Columbia, the diversity of school district A B E programs, the concept of educational accountability, and the political economy of provincial policy development and implementation are shown to contribute to conflicted understandings of the purpose of an A B E accountability framework. The dynamics involved in this policy process underscore the often circuitous nature of government policy. ABE is an attempt to address the learning needs of adults at risk of economic and social marginalization and provide them access to further learning opportunities. The structure and quality of adult education is not only thought to influence the economic growth of the country but also promote individual inclusion and active citizenship (Statistics Canada, 2001). It might therefore seem reasonable to assume that intrinsic to the design of A B E accountability initiatives would be a strategy to encourage participation (ibid.). In British Columbia participation is promoted by providing adult basic education through both the college and school district 2 systems. While increasing opportunities for access, the dual ownership paradigm has also resulted in a politically charged environment shaped by historical arrangements and a changeable, if not volatile, social, political, and educational milieu. These dynamics guided the development of the accountability framework addressed in this thesis. They impacted my experiences and understandings which are located in school district delivery of ABE, but intimately tied to the context and background of A B E in BC. In British Columbia the Ministry of Education (MoE) provides school districts with funding for adult basic education which is understood to be coursework that prepares adult students to complete their high school graduation. The problematic nature of A B E accountability surfaced in the Spring of 1998 when the Minister of Education concluded that the graduation rate for participants of school district A B E programs was only 8.3%. Based on this information the minister commissioned a province-wide survey of school district A B E programs (see Reed and Associates, 1998, for findings of the Educom survey). The objective of the survey was to gather information that would lead to a more universal, efficient, and learner-centred A B E system. The survey results demonstrated there was little consistency across provincial A B E programs with regard to accountability practices. It also surfaced inadequacies within existing MoE policy. In 1998 school districts were permitted under MoE guidelines to grant adult graduation diplomas without reporting them to the ministry. The presupposed 8.3% graduation rate that instigated the survey had not included graduates granted diplomas by school districts. The survey suffered from certain limitations due to inconsistencies in the way the data were gathered. School districts reported course completion and graduation rates subjectively, which 3 contributed to limitations within the data. These inconsistencies prompted the MoE to move forward on a number of initiatives designed to enhance accountability within school district ABE programs while recognizing and supporting the unique needs of A B E students. This led to the development of an accountability framework for MoE funded A B E programs. From 1998 to 2001 the MoE was engaged in examining the accountability practices of adult basic education offered through the school districts of British Columbia, and in the final two years of that timeframe, developed in collaboration with the field a comprehensive student-centred accountability framework responsive to the diverse needs of government, school districts, and adult learners. Implementation of the framework was stayed due to the NDP government losing the provincial election to the Liberals in 2001. By the Fall of 2004 the Liberal government had introduced an economically driven funding policy for ABE that failed to acknowledge the recommendations of the ABE accountability framework. Accountability and subsequent funding would now be determined on the basis of course completion rates. The 2004 funding formula is still currently in use. Its implementation departed from past practice that tied funding to the number of courses students were registered in rather than the course completion rate. It is a crude policy tool that does not recognize disparities in school districts which may impede course completion, such as access to public transportation and ethnicity. It is not student centred and does not reflect the often complicated socio-educational backgrounds of many A B E students. Neither does it reflect the multiple responsibilities of adult learners. An ABE accountability policy based solely on course completion rates has the potential to become a barrier that financially limits school district ABE programs and possibly denies access to many adult learners who are, by virtue of circumstance, marginalized and excluded. 4 The Ministry of Education's primary mandate is the provision of a K-12 educational system for students under the age of nineteen. The MoE's adult basic education program has come under scrutiny as the ministry looks for ways to reallocate existing monies from this historically, politically, and educationally controversial sector of the public education system to the regular K-12 system; this is especially true in the current political climate. To facilitate the MoE's continued funding of A B E programs requires that school district providers have an accountability framework that addresses the unique circumstances of adult basic education and supports improvement of school district A B E programs. Such an accountability framework would provide the ministry with a reliable process to evaluate how A B E providers utilize ministry funding for the development and implementation of educationally responsible A B E programs for adult learners. The framework would also provide a policy mechanism to demonstrate how school district ABE programs are successful and accountable. Accountability has become a ubiquitous construct within the rubric of K-12 education. Politicians welcome research that yields data on the difference between low and high performing schools, which are generally judged on the basis of standardized test results. When taken out of context, standardized test results have the potential to further marginalize A B E students. In British Columbia right-wing think tanks such as the Fraser Institute use provincial test results to fuel a politicized interpretation of school performance, and steer public and political opinion in predetermined directions that define education through a very narrow test-driven lens. This has the potential to cocoon the British Columbia K-12 system and school district A B E programs in a philosophic doctrine whose educational and social purpose is dominated and manipulated by economic and political influence. 5 Test-driven approaches to educational accountability have not significantly impacted the provincial high-school dropout rate, which has hovered around 30% for decades. The intractable nature of the dropout rate would seem to suggest that traditional test-driven measures of K-12 accountability neither result in school improvement nor adequately reflect the impact of input factors such as teaching approaches, curriculum design, and socioeconomic situations. A more holistic approach to educational accountability would reflect the context of the program and incorporate indicators of success that were program specific and not limited to standardized tests. A preferred accountability model would incorporate both qualitative and quantitative data to assess a range of indicators of program and student success. Program level success would reflect outcomes as well as inputs such as financial resources with indicators possibly including allocation of resources, student-teacher ratio, teacher competence, and physical environment. Indicators of student success would include demonstrable acquisition of a range of psychosocial, employability, and organizational skills. Academic indicators of success could be measured using nontraditional assessment such as portfolios combined with test scores and course completions. The combination would be determined at a district level and provide performance indicators that reflect the learning profile of adult students, the educational needs of adult learners, and the needs of the community. Through these indicators the district would be able to demonstrate how it is currently allocating resources to promote adult learning programs and suggest how it could improve the future delivery of existing programs. The development of an accountability framework for ABE is a complicated process given that all fundable adult students enrolled in Ministry of Education A B E programs are school leavers, who for a number of reasons are not able to achieve success within the normal structures of the 6 traditional K-12 system. They often bring with them personal problems that interfere with participation and performance and they are once again, as in past educational endeavors, at risk of not finishing school. The Purpose of this Study The purpose of this research is to develop a deeper understanding of issues related to accountability in adult basic education. This will be accomplished by examining in retrospect my first-hand involvement in a specific attempt to develop an accountability framework for publicly funded adult basic education programs in British Columbia. Between 1998 and 2001, the British Columbia government's Ministry of Education was engaged in a process of determining how accountability could be assessed for the ABE programs offered through the province's school districts. In the final two years of that tenure, the MoE's A B E accountability committee constructed a framework that was intended to address the needs of adult learners, the school districts' ABE programs, and the ministry. I was seconded by the MoE to participate in the entire process. From this perspective, the dissertation describes my experience of being involved in the design of the framework throughout its three-year evolution. I then provide an assessment of the framework's strengths and limitations as a tool to represent the multiple outcomes of school district adult basic education programs in BC, and examine the possible reasons why the framework was not subsequently implemented. In conclusion, 1 summarize what we have learned in general about accountability as it applies to adult basic education from looking at this particular instance, discuss implications for the future, and suggest areas for further research. 7 The Research Question One broad question guides the study: What influenced the development and fate of the ABE framework? This question will be addressed by examining the affect the political economy had on the process, the product, and the framework's denouement. Political economy is defined as the social relations or power dynamics that exist between political, economic, and educational interests participating in decisions affecting the framework's development, evaluation, and end result.2 My Relationship to the Study This study reflects on my lived experience and direct involvement in crafting, developing, and testing a province-wide A B E accountability framework for the school districts of British Columbia. As a result of the 1998 Educom survey conducted by Reed and Associates (a private consultancy), and in an effort to better understand school district delivery of adult basic education, the Ministry of Education launched a search for an experienced school district senior administrator in adult education who would facilitate a review of school district ABE programs. As the successful applicant I was seconded to the MoE as the Provincial Coordinator for Adult Education from August 1998 to August 2001. My primary responsibilities were to liaise with representatives in the field and the ministry to ensure a more student-centred delivery system for adult basic education in British Columbia. 1 worked with representatives from the MoE and the Ministry of Advanced Education, Training and Technology (MAETT) to develop a more coordinated and accountable system of ABE delivery in British Columbia. During my secondment, I served as the provincial co-chair of 8 cross-ministry subcommittees that oversaw policy related to the delivery of A B E programs. In the final year of my secondment I also served as the provincial chair of the MoE committee mandated to develop and implement an ABE accountability framework for school districts. In January 2000, in response to the MoE's desire to move forward with a provincial accountability framework for A B E and the ministry's need for a more comprehensive understanding of how school districts provide academic basic education for adult students, I initiated a provincial review of district A B E providers. The review revealed differences in school district delivery of A B E that were impacted by, but not limited to, factors such as regional demographics, teacher certification and qualification, hours of operation, delivery format (i.e., traditional or nontraditional), physical facilities, course selection, and curriculum delivery (i.e., teacher-driven or computer-driven, or both). The provincial review enabled me to better understand the context of A B E delivery as I worked with the ministry in the development and implementation of an A B E accountability framework to reflect the diverse nature of school district ABE delivery. The construction of the framework was guided by a Ministry of Education ABE accountability committee which included two senior administrators with school district ABE programs and two MoE officials who oversaw MoE adult basic education in BC. I was one of the two senior administrators responsible for school district A B E programs seconded to serve on this committee. My involvement included responsibilities for gathering research data, interacting with stakeholders, and advising the MoE accountability committee as one of its members. In September 2001 I returned to my school district and resumed my role as Director: Adult and Continuing Education. I am employed by a rural school district that enrolls approximately 9 10,000 students. In 2002 the adult basic education program enrolled approximately 1,000 full-and part-time adult learners of which some 25% were First Nations students. The district covers an enormous region (373,000 hectares), with a population of approximately 72,000 scattered through a number of small communities. I was hired by the school district in 1995 to design and implement an adult basic education program. To best serve the educational needs of the adult community I established one central adult learning centre and three satellite centres in the neighbouring communities. The adult learning centres offer the provincial adult graduation certificate (Adult Dogwood) and the regular graduation certificate (Dogwood). In June 2002 I was elected President of the British Columbia School District Continuing Education Directors' Association (BCSDCEDA, or CEDA, as it is more generally known), an association of school district continuing education administrators that meets regularly with the MoE to discuss issues concerning the school district delivery of A B E programs. ABE is one of many adult programs that school districts offer, but it is the only adult program funded by the Ministry of Education. Approach to the Study This thesis works within the "focus on practice" paradigm that differentiates the EdD from most conventional doctoral studies and is one of the main features that first attracted me to the program. The story that unfolds infuses the policy decision-making events with self-reflective and self-actualizing implications that arise from being a practitioner-researcher involved in the development of the A B E accountability framework. A subtext to the policy narrative is thus an introspective and critical analysis of my experiences and understandings as an educational 10 practitioner in a political arena. My professional experience with school district ABE programs and in the MoE necessarily impacted the approaches 1 engaged in to track and interpret the evolution of the accountability framework. This section delineates these approaches: the material sources that informed the research, the observation processes used, the steps taken to analyze, interpret, and organize the findings, and the assumptions, or biases, 1 brought to the study. This study characterizes A B E as an innovative educational program that is specific, complex, and functioning. Educational accountability within A B E programs is the object of analysis. The setting is the province of British Columbia. The context is school district adult basic education programming. The scope is bounded by a specific time period (1998-2002). As a review of a past policy decision bounded by time, place, and context, the analysis takes a retrospective case study approach, as this is the most appropriate means to piece together data, convey information, create knowledge, and better understand the particular phenomenon (Stake, 1995; Yin, 1994) of educational accountability within the context of adult basic education programs. Sources of Evidence The analysis is informed by multiple sources of evidence, including ministry documents, academic literature, and field notes documenting my impressions of conversations and meetings that I held with stakeholders and participants involved in the design, development, and field testing stages of the accountability framework. Documents were used to corroborate and augment evidence from other sources and to make inferences. As a school district senior administrator seconded by the MoE, I had privileged access to confidential primary data, including ministerial briefing notes, documents, and archival 11 records. In addition, I had open admission to the minutes of the varied and innumerable committee meetings held during the evolution of the accountability framework which were recorded and transcribed by an administrative assistant in attendance. Primary data in public domain included policy reports from the BC provincial government, US policy documents, and the very few pertinent OECD publications addressing ABE. One particularly relevant document was a provincial review of ABE programs that included direct observation and informal conversations with field practitioners throughout BC (Moss, 2001).3 Secondary literature was used as a tool to discern where others had and had not gone in the field. A literature review demonstrated the paucity of published research specific to the purpose of this study. This lack of related research highlighted the relevance and potential significance of this study (Massey, 1996). What literature there was enabled me to compare assumptions with previous research and helped me focus on the problem, questions, and justification for the study. It informed my input into the design of the ABE accountability framework and assisted with building my argument to support the framework as a long-term policy solution. The literature review also provoked thinking in certain directions such as governmentality and professionalization which helped me in the analytical stage. A rich source of information was derived from the field notes that I took, which recorded my experiences as a reflective practitioner who assumed a number of roles in the framework's design and fully engaged in the events under study. Although analyzing a process in which I had some vested interest could be criticized for its potential to introduce more biases into the study, it provided an intensity of experiential understanding and first-hand knowledge that I could not have gained otherwise which supported aspects of the framework's design and surfaced 12 unanticipated outcomes. As a member of the ABE accountability committee which was charged with drafting the accountability framework, I met with A B E teachers and school district senior managers throughout the design process to ensure the framework was not limited by or precluded from the expertise of interested groups or individuals. The ABE accountability committee met formally for two years on a tri-monthly basis to review external research on accountability and confer on how the development of the framework could best proceed. I served on the committee for the entire duration. Between meetings committee members clarified issues by meeting with stakeholders which included field practitioners (teachers and school district A B E program administrators) and ministry officials. I held innumerable conversations about the design of the framework on a daily basis with these stakeholders. Whether from the ministry or the field, these discussions engaged knowledgeable people who had responsibility and potential to influence the framework's design and possible implementation. My field notes record understandings gleaned through these exchanges and are relied upon here as general impressions. The insights gained in the process were instrumental to the research reviewed in this study and in some instances they guided the inquiry. The completion of the framework's design meant the end of my secondment to the Ministry of Education. It resulted in an exit report to the ministry regarding A B E accountability and the design of the A B E Framework. The exit report contained the MoE accountability committee's recommendation that the framework be piloted at the school district level. Specifically, the report recommended that Based on exhaustive discussions concerning research into A B E accountability within the committee and in conjunction with the field, the MoE committee on ABE accountability resolved the framework be considered for piloting in selective 13 districts. The draft proposal of the framework suggested in this report was reviewed and supported by a number of school districts and although it does not address all the aforementioned A B E issues [surfaced in the provincial review of school district ABE programs], it should provide an adequate framework for field testing. Information gleaned from the field tests should provide the ministry and the field with the necessary refinements to better ensure the success of a provincial framework. (Moss, 2001, p. 13) Discussion regarding the length of the pilot process, although not definitive, resolved that four to six months would be required to properly pilot the A B E framework. Due to a change in government a formal pilot of the framework was never completed; however, as a result of the recommendations in the July 2001 exit report on A B E accountability one school district ABE program conducted an informal review of the framework to assess the mechanism's effectiveness at evaluating the quantitative and qualitative dimensions of adult basic education.4 The informal review extended over five weeks in April and May 2002. From conversation with those involved in the review, I gleaned invaluable insights into the framework's strengths and limitations for representing the many diverse outcomes of school district A B E programs. I also acquired foreknowledge of the possible pitfalls the framework's implementation might anticipate at the district level and gained deeper insight into the unique, localized nature of A B E accountability. Feedback through general discussion with members of the district committee contributed to a pool of impressions regarding the efficacy of the accountability framework which was distilled and is represented in Chapter Five (Table 5.1). At the conclusion of the informal school district ABE accountability framework review, participants met members from the MoE A B E accountability committee to discuss the strengths and limitations of the framework evident from the district's critique. Members from the MoE's accountability committee then provided a written summation of the concluding meeting with the school district review participants which finalized the district review and contributed to the findings analyzed in this study. 14 Reflection, Organization, and Interpretation A retrospective analysis followed the stages of information gathering. Retrospective case studies are a cost effective, time efficient, and non-intrusive means of constructing empirically based research. A traditional case study that interjected a tape recorder and formal interviews into the dynamic might or might not have provided better data but it would definitely have caused unacceptable delays in what was at base a government research project constrained by time and confidentiality requirements. Stakeholders might also have been less forthcoming in sharing their knowledge, thus reducing the quality of the findings. Although retrospective studies are somewhat limited by the researcher's memory and reflective ability, these constraints may be offset by diligent reliance on well-organized field notes and by "member checking" (Stake 1995: 115), which in this case meant having an A B E administrator from another provincial school district and a ministry official familiar with the framework's evolution critically review a draft of this dissertation to verify the accuracy of its interpretations, findings, and inferences. The strategy for understanding and interpreting the multiple sources of information involved iterative reading and cross-referencing of academic research, theoretical literature, ministry reports, transcripts of committee meetings, and field note data recorded during exchanges with stakeholders. This strategy revealed different understandings of criteria that constituted success in school district A B E programs. It enabled A B E success to be viewed through a variety of lenses to present a holistic interpretation of historical, socioeconomic, cultural, geographic, attitudinal, and behavioural issues. This process combined with member checking as outlined above was central in corroborating my sources of information and interpretations to enhance the efficacy of the conclusions reached, and hopefully to yield deeper understandings of the results posed by the research questions (Creswell, 1998). 15 The process of examining my field notes and the reflections and understandings that resulted were influenced by Stake's (1995, p. 77) guidelines for research. This involved the search for patterns by disaggregating the information, organizing it in terms of themes, ideas, and arguments, re-sorting it according to its affinity with other recurrent themes that emerged in the literature and during various committee processes, then recombining, consolidating, and compressing these concepts in congruence with descriptive categories that appeared to provide the most meaningful interpretations of how educational success might be defined through a variety of different filters. Particular attention was paid to themes of socialization, professionalization, governmentality, reification, opportunities for emancipatory practice, and signs of resistance that I recognized through prior reading of the literature, which sensitized me to observing concepts and phenomena that I may not have previously been aware of or realized were important. This process of interpretation and understanding is a highly personal reflection of the researcher's work ethic, knowledge, life experiences, and tendency toward reflexivity (ibid.). In my case the process of making meaning was interpretive and systematic but not linear or orderly. Preconceived Ideas Brought to the Study Every researcher brings assumptions, interests, and biases to their work which guide the selection of research questions and the decisions taken to prioritize and report the findings (Creswell, 1994). I am no exception. My first assumption was that the goals and objectives of ABE would be difficult to measure and open to interpretation. Furthermore, I knew that research related to A B E and accountability in BC was limited from having done the literature review. As the researcher, I interacted with what was being researched. Description of the design and 16 analysis of the ABE accountability framework was influenced by my experience as a Director of Adult and Continuing Education in a provincial school district. It was also influenced by my experience as a secondee to the MoE responsible for reviewing school district A B E programs and as a member of the committee guiding the design of the A B E accountability framework. The findings are admittedly value-laden (Creswell, 1998), although I made every effort to acknowledge my personal bias in the analysis and reporting of findings. My second assumption was that different stakeholders might have different points of view and perspectives with regard to their understanding of educational accountability. Their views might depend on their understanding of the purposes of A B E and accountability and how success could be most effectively measured within those understandings. For the purpose of this study, the key stakeholders were the MoE and school district providers of ABE. While each stakeholder group held objectives for A B E accountability that were significant to this study, these objectives were assumed to be different. I thought the MoE's understanding of A B E accountability would tend to be based on student success measured through academic indicators such as completion rates, while school district providers of A B E would tend to understand student success through academic indicators and non-academic indicators such as self-confidence. A third assumption was that the stakeholders' perspectives would reflect their sociopolitical status and realities. The Ministry of Education's perspective might be economically driven while school districts might be more educationally focused. These conflicting perspectives would reflect attitudes that needed to be considered in the design of an accountability framework. The significance of these realities is underscored by the fact that the MoE was the sole funder for school district A B E in BC. 17 My fourth assumption was that over the long term both First Nations and immigrant populations could be expected to increase significantly throughout the province. As differences among cultures, ethnic groups, and other demographic traits accelerated, we might anticipate that the considerable divergences and disparities already existing across the school districts of BC would rise substantially. A major component of the framework's effectiveness would therefore be its capacity to be flexible and sensitive to the diverse and evolving socioeconomic and cultural factors that impacted upon the needs of adult learners residing in each district. The fifth assumption was that school district teachers working on the frontlines with adult learners would be the most able to assess and explicate the shifting needs and diverse signs of achievement demonstrated by the populations they served. If the framework was to be effective, the people responsible for following accountability guidelines would need to be part of the process for determining what accountability was, how and why such a framework might be implemented, and for suggesting innovations to the framework to better assess emergent areas of practice required to meet the variable needs of adult learners. This would require that notions of accountability be built upon the value systems of practitioners and that praxis become a central resource on which to base effective education policy decision making. A sixth assumption was that the understandings arising from this study would be forged by the readers, as they perceived similarities between these findings and their own situations. The interpretive nature of this study would have different, highly particularistic degrees of significance for different localities dealing with the accountability of A B E programs. 18 Delimitations and Limitations This research is confined to an analysis of the development of an A B E accountability framework by the MoE in British Columbia over a three-year timeframe, from September 1998 to June 2001. The design and development of the framework was problematic because there was limited research related to accountability within the delivery of school district A B E programs. In the development phase, the framework was informally reviewed by four districts, and although the framework underwent an abbreviated five-week critique by one large district in the Spring of 2002, it did not undergo the necessary four month minimum the MoE accountability committee deemed necessary to formally pilot the mechanism in a British Columbia school district. Formal piloting of the framework to this extent and its eventual implementation did not occur due to a change in government following the 2001 provincial election. The research provides an overview of BC school district A B E programs and offers insight on the topic of educational accountability within these programs. As there has been limited research to date on the subject, the study may stimulate considerable discussion, but its conclusions should be viewed conservatively. The analysis was subject to the attitude and perception of the researcher, and although not intended, the biases of the researcher as a program advocate may have influenced the interpretations. The Study's Significance An analysis of the design, development, and non-implementation of the A B E framework constructed for school district providers in British Columbia is important for several reasons. Morally, there is a societal responsibility to provide educational opportunities that enable 19 individuals to recognize and take initiatives to change their life circumstances. Research indicates that students who have not obtained secondary graduation are less likely to find employment and more likely to be socially excluded (Klasen, 2000). Social exclusion impacts an individual's ability to participate effectively in economic, social, and cultural life (ibid.). ABE is a nontraditional educational pathway for learners who have been unsuccessful in the traditional K-12 system. For many adult learners ABE is the "only game in town" (Mezirow, Darkenwald, and Knox, 1975). The design of an A B E accountability framework should therefore support the efforts of the students who attend these programs. The Province of British Columbia has a new welfare-to-work mandate. In an effort to reduce the number of individuals on welfare, the policy limits the time recipients can collect welfare. As a director of A B E programs, it has been my experience that those with the least education are the most likely to be on welfare. In an era of welfare reform, education is important for individual economic self-sufficiency (D'Amico, 1999). A B E provides educational pathways for individuals who lack basic skills and who, without an education, would be unable to find employment. If the government was sincere in its desire to assist individuals in moving from welfare to work, it would ensure continued support for A B E programs through an accountability mechanism that recognized both qualitative and quantitative indicators of program and student success. Structure of the Thesis The rationale for this study, my situated insights and possible areas of bias, as well as the processes undertaken to conduct the study have been presented here. The chapters that follow from this one delineate the research findings and conclusions. 20 Chapter Two provides an abbreviated policy history of adult basic education in British Columbia. Although government support for adult education programs predate Confederation, this history begins by tracing federal initiatives to provide adult literacy training in the 1960s and the provincial government's involvement in A B E from the mid 1970s. It addresses the development of system differences in A B E programming among the school district and college sectors during the 1980s and 1990s. The chapter ends in 1999, the year the Ministries of Education and Advanced Education, Training and Technology, in collaboration with representatives from the field, attempted without success to initiate a common accountability framework for A B E programming in both the college and school district sectors. The inability of all sectors to reach an equitable agreement led the Ministry of Education to move forward with the construction of a province-wide ABE accountability framework for its school districts. Chapter Three provides an integrative research review of concepts that shed light on the development and effectiveness of the framework, as relayed in the remaining chapters of the study. Focus of this chapter is on the social, political, economic, and educational purposes of educational accountability, the structural and human dimensions to consider when designing an educational accountability framework, and the power dynamics shaping the tone and tenor of accountability initiatives. The review lays the conceptual basis for subsequent chapters while weaving central tenets from the literature into an argument supportive of the development of an accountability framework that legitimizes program success through both qualitative and quantitative lenses. Chapter Four outlines the planning and construction of the A B E accountability framework for the Ministry of Education's school district sector. While the framework's development was 21 informed by literature and past provincial policy, what makes this structure particularly unique are the field research findings of a province-wide school district review that disclosed the state of ABE accountability needs in the K-12 system. These combined data sources contributed to the construction of an accountability framework able to acknowledge the many ways program success might be demonstrated and measured so that it would be adaptable to demographic, economic, and cultural changes and sensitive to local differences, learner needs, and political priorities. The chapter concludes with the ABE accountability framework that resulted from the two-year process of construction through research, committee work, and consensus building. Chapter Five focuses on my impressions of the framework's robustness which were derived from feedback of participants conducting a five-week review of the mechanism's efficacy. My inferences were formed through communicating with a group of A B E practitioners who were charged with evaluating the framework's capacity to address the programmatic issues identified by the province-wide district review as relevant to their school district and to ferret out information on nontraditional indicators of student progress. The exercise deconstructed conventional interpretations of accountability to the point of uncovering highly unexpected and desirable transformational possibilities and emancipatory qualities that were deeply embedded in the concept. Chapter Six analyses the study's major findings through a lens focused on the power dynamics of different politicized interests. It provides a retrospective, self-reflective analysis that is theoretically grounded and situated in the contemporary context. As I reflect upon my experiences as an education practitioner in a political arena, I speculate on the possible underlying motives behind the government's decision to instigate, then abandon, the 22 construction of an A B E accountability framework, and whether the outcome would have been different if there had not been a political regime change. I then describe the possible impact of the Ministry of Education's decision under the aegis of the current government to turn to test-driven and course-completion modes of measuring accountability. Chapter Seven summarizes the study and articulates the key findings of the research. The focus is on the implications of these findings for the field of adult education. I conclude by offering suggestions to future architects of educational accountability initiatives on how they might effectively manoeuvre through this highly politicized landscape to meet with more receptive ends. I then provide recommendations for further research. 23 Notes 1 A glossary of terms commonly used in this dissertation such as A B E is provided at the beginning of the thesis for the reader's convenience. 2 This definition of political economy is an adaptation from Drazen (2000). 3 Although prepared by me, this was the only document at the time this research was conducted that contained information/data directly relevant to school district delivery of A B E programs. Due to the field review of various districts and discussions with staff, the data gleaned in this document became a launch point for the need for a common accountability system that would recognize regional and district differences. School district A B E had never been reviewed, and the issues raised as a result of this review flushed out many accountability problems that needed to be dealt with to legitimize continued funding for ABE. There were other documents that discussed and defined ABE, but there was nothing specific to school district delivery of ABE. 4 Quantitative dimensions were defined as traditional academic indicators such as standardized test scores, course completion and attendance rates, teacher qualification standards, and the proportion of school district A B E funds allocated to school district ABE programs. Qualitative dimensions were understood to include non-academic indicators such as changes in behavioural patterns manifested by self-confidence, competence at meeting deadlines, signs of respect, patience, initiative, and determination, and the capacity to ask for help and work with others. 24 Chapter Two The Context: Adult Basic Education in British Columbia This chapter traces the evolution of government policy for adult basic education (ABE) in British Columbia from the 1960s on. It briefly notes the arrested development of a federal policy for ABE in Canada, then focuses on British Columbia's ABE policy history from the mid 1970s to the late 1990s, when attention shifted toward building an.accountability policy for the province's publicly funded A B E providers in the school district and college sectors. In BC this segment in the history of government A B E policy can be characterized as one of interministerial and intersectoral problems that for several decades discouraged the design of a unified provincial approach to the delivery of A B E programs. This examination of the development of A B E policy in British Columbia will help put into context the reasons why accountability in the delivery of school district adult basic education programs is of interest at this time. Background Extremely intermittent forms of government funding for adult education in British Columbia can be traced to the 1860s (Selman, 1977). More consistent public funding specifically addressing ABE began with the involvement of federal government policy, which developed in Canada and the United States almost simultaneously during the 1960s. At the time both countries were increasingly becoming the home of waves of immigrants whose first language was not English, or, in the case of Canada, neither English nor French. Adult basic education has many distinctly regional definitions, but broadly speaking both Canada and the US would have it include "everything from literacy education through to high school completion for adults", with 25 education beyond the literacy level being "normally related to [the acquisition of] formal educational credentials" (Selman and Dampier, 1991, p. 279). In the US, the Adult Education Act of 1966 formally established an ABE program. This was the first of several federal initiatives undertaken by that country to assist adults "whose ability to speak, read, or write [in the] English language constitutes a substantial impairment to their ability to get or retain employment commensurate with their real ability" (Darkenwald and Merriam, 1982, p. 178). The legislation aimed to help eliminate that barrier, raise the students' level of education, improve their capacity to benefit from further training, increase their opportunities for profitable employment, and make them more able to meet their responsibilities as adults and citizens (Mezirow, Darkenwald, and Knox, 1975, p. 3). In the US funding and accountability for adult basic education have remained federal responsibilities under the aegis of the Department of Education. Similarly, in Canada, adult basic education, as an element of formal government policy, developed in the 1960s in response to rapid social and technological change. Growing demands for social equality, heightened immigration, a "red scare" based on the Soviet Union's technological prowess, and an emergent Canadian nationalism contributed to increased federal investments in the technical and vocational training of adults (Shohet, 2001). The earliest federal initiative was the Technical and Vocational Training Assistance Act of 1960 which made funds for training and capital assistance grants available to the provinces. Attempts to implement this legislation revealed that many adults did not have the sufficient basic literacy or numeracy skills to participate in vocational training opportunities (Selman and Dampier, 1991). This led to a spurt of federal-provincial adult basic education initiatives being implemented in the 1960s. 26 The federal government introduced the Adult Occupational Training Act in 1967 as a means to provide unemployed and underemployed adults with the basic skills needed to participate in vocational training programs. As part of the act, the Basic Training for Skills Development (BTSD) program was launched to provide adult basic education at the K-12 levels. This was followed by the Basic Job Readiness Training (BJRT) program in 1973, which offered short-term courses on life skills and job search techniques as well as work experience programs that were intended to lead adults directly into the workforce (British Columbia, Ministry of Advanced Education, 2005; Shohet, 2001). Federal initiatives in A B E program design came to an end by the 1970s, as an economic recession led to retrenchment in federal investments in ABE. Accountability studies conducted in that decade indicated the BTSD and BJRT programs were not meeting expectations (Shohet, 2001). Jurisdictional disputes were reaching an all-time high over several federal-provincial portfolios, natural resources especially, but education as well. Due to Canada's constitutional arrangement in which education at all levels was deemed to be a provincial responsibility, the Canadian government's involvement in A B E was precluded from ever becoming as deeply entrenched or well articulated as it was in the US. By 1982, at the height of Canada's recession, federal funding in support of A B E was cut and responsibility for funding shifted to provincial and private sponsors (Selman and Dampier, 1991, p. 279). At this time, one observer noted, "provision for the most undereducated adults had almost ceased to exist" (Thomas, 1983, p. 65). While the federal government continued to support ABE initiatives, its role became one of facilitator of provincial initiatives rather than an initiator in its own right (Shohet, 2001). 27 Each Canadian province has a distinct history tracing the development of informal and formal programming for adult education. In British Columbia, highlights of this history include the first government grants to adult education initiated by the Government of the Mainland Colony in 1865 (which were discontinued in 1868), the Vancouver School Board's launch of the first comprehensive night school program in 1909, and the same Board's creation of a full-time centre for adult learning in 1962 when it turned King Edward Secondary School into Vancouver Community College, which set off a pattern of college formation out of school board continuing and adult education activities.1 Despite these early and generally local initiatives, however, adult basic education would only become a focus of the provincial government in the 1970s. In British Columbia, up until 1986, a single ministry, the Ministry of Education, was responsible for formal education at all levels, from K-12 through postsecondary. In 1976 the provincial government called for the MoE to participate more in the development of continuing and community education programs in its inaugural report on ABE, Helping to Develop a Provincial Continuing and Community Education Policy, which was released in September that year (British Columbia, Ministry of Advanced Education, 2005). In December 1976, a committee on continuing and community education in BC released another report which called for ABE to become a "high priority special program". In 1977 the Colleges and Institutes Act legislated that colleges were independent of school boards, yet some boards retained control over their continuing and adult education programs, which initiated competition between the two systems. Some five years later, in 1982, the MoE released the report Ministerial Policy on the Provision of Adult Basic Education Programs including English Language Training in the Public Education System of British Columbia which made it the ministry's responsibility to provide adults access to high quality ABE programs (ibid.). It also delegated responsibility for the development, 28 administration, and delivery of A B E programs to the colleges, institutes, and public school system. By 1983, the process of articulating ABE courses had begun. Colleges established province-wide articulation of A B E programs, while school districts, though offering A B E programs, chose not to engage in the process of articulation (British Columbia, MAETT, 1996). In 1986, responsibility for A B E was transferred from the MoE to the newly created Ministry of Advanced Education (MAE). Three years later, at the inception of the Access for Al l policy, the provincial government urged school districts to become more involved in A B E programming. Thus by 1989, with the support of MoE funding, school districts began to provide courses toward high school graduation on a tuition-free basis for adults who did not have a secondary school graduation credential. This policy exacerbated the existing jurisdictional conflicts between the two ministries. Currently, legislation in British Columbia permits school districts to provide provincially funded ABE to adult learners while it requires colleges to provide this function. Both systems deliver ABE programs that lead to grade 12 graduation or serve to upgrade a student's academic transcript. Both offer adult learners the prerequisites, credentials, and skills for participating in further education and the job market. That is where much of the similarity ends, however. System Differences School district A B E programs fall under the aegis of the Ministry of Education which under the K-12 School Act enables school boards to "permit a person who is older than school age to attend an educational program in accordance with any terms and conditions specified by the 29 board" (Selman and Dampier, 1991, p. 157). By contrast, college ABE programs are the responsibility of the Ministry of Advanced Education (sometimes known as the Ministry of Advanced Education, Training and Technology, or MAETT, during the timeframe of this study) and are guided by the Colleges and Institutes Act which states that the object of a college or university college is to provide comprehensive adult basic education. Not only do the college and school district systems operate under different mandates but their mandates are directed toward different considerations. Colleges are legislated to provide learning opportunities for adults to develop some or all of the skills required to function successfully in Canadian society (Selman and Dampier, 1991). School district ABE providers, on the other hand, are permitted to offer opportunities for adult students to acquire their grade 12 graduation certificate or academic upgrading courses. The differences in the A B E programs offered by the two systems are a reflection of their different mandates. Even though colleges offer the same Adult Dogwood high school diploma as school districts, they are not restricted by a standardized curriculum designed for children and young adolescents, as are the school districts. The college system has far greater flexibility in the development of coursework leading to the grade 12 credential. School district providers of ABE are restricted by the Ministry of Education's K-12 curriculum and accountability structures that do not reflect the characteristics of adult learners or the multidimensional goals associated with ABE. Instead, the school districts must use curriculum that is predicated on content and outcomes that are written for school-aged K-12 learners. For some A B E students, this acts as a dispositional barrier to participation and contributes to student attrition. 30 The college system is funded through the Ministry of Advanced Education, and prior to July 1, 1998, it granted a Provincial Diploma as a graduation credential. The Provincial Diploma consisted of five courses and included the college equivalent of English 12, Math 11, and three other grade 12 level courses. School districts are funded by the Ministry of Education and grant two graduation credentials. Prior to July 1998, the Adult Dogwood consisted of seven courses at the grade 11 and 12 levels, which included requirements for an English 12, a Math 11, Social Studies 11 and four other courses at the grade 11 and 12 levels. Districts also grant the full credit Dogwood which is the traditional thirteen-course graduation credential associated with the Ministry of Education's K-12 mandate. Although the Provincial Diploma was classified as a grade 12 equivalency, the Ministry of Education Dogwood has always been better recognized by the marketplace as the traditional British Columbia graduation credential. From 1983 to 1989, colleges were the main providers of ABE. In 1989, the Ministry of Education agreed to provide full tuition for adults wanting to return to school to complete their graduation credential. This decision was fuelled by the 1988 report of the Royal Commission on Education which recommended that school districts incorporate the concept of lifelong learning in the planning and delivery of educational programs. A false distinction cast by the decision was that colleges came to be thought of as the ABE providers whose primary responsibility was to offer upgrading for students wanting to enter postsecondary programs, while school districts began to be viewed as the A B E providers for students seeking a graduation credential. The distinction had tangible effects. Students enrolled in college A B E programs paid tuition of approximately $90 per month (Faris, 1992), whereas school district A B E students paid no tuition. 31 The dual provider circumstance began to create problems over "ownership" of ABE, and ultimately, over whose responsibility it was. Colleges considered themselves more adult-oriented than school districts, while school districts argued that their mandate was a grade 12 graduation pathway for all learners, including adults. They demonstrated that they too could accommodate adults with adult-oriented learning environments as well as a curriculum that fulfilled Ministry of Education learning outcomes and granted adult students the Dogwood graduation credential. The distinction between the two sectors carries over into the professional qualifications of ABE teachers. In neither the school district nor college sectors are A B E teachers required to attain a professional credential demonstrating formal or informal competencies in adult learning theory and educational practice. Although each postsecondary institution in British Columbia has its own employment standards, a review of ABE instructor qualifications required for job postings in the province's college sector indicates a baccalaureate is usually required at minimum with preference granted to those holding a master's degree, ideally but not necessarily in the discipline being taught. While experience in teaching adults is often a prerequisite, it would seem that many trainers in the postsecondary sector do not have teacher training. The college sector has gone some distance, however, to improve instructor qualifications by strongly encouraging all vocational instructors, including those of ABE, to complete an adult learning certificate that reflects formal theory in adult education and assists instructors in developing curriculum and better understanding how to teach adults. In general, school districts are prescribed by the BC Teachers' Federation to hire professionally certified teachers (which tends to result in their paying higher wages than the college sector offers). While most of these have similarly not engaged in specific professional training in adult learning theory and research, they have been 32 certified by the BC College of Teachers as having completed the necessary academic and practical requirements for teacher certification in the province's K-12 system. Although both sectors do not specify prerequisite training in adult learning theory, the BCTF and the FPSE have worked hard to enhance the efficacy of ABE instruction. The BCTF introduced its Adult Educators' Provincial Specialists Association in 1997 and the FPSE has continued to work to overcome government cut-backs to A B E and increase awareness with regard to the professionalization of A B E instructors. The heightened emphasis that both sectors have placed on the professionalization of A B E instructors marks a significant departure from past practice (at the district level I am aware of some districts in 1996 employing A B E instructors with little formal training - in some incidences their qualifications did not exceed grade 12).3 Thus in British Columbia, neither the college nor K-12 sector can lay claim to A B E on the basis of their superior professional knowledge, experience, or credentials. Both sectors, through their professional organizations, have worked diligently to enhance the professional qualifications of ABE educators and demonstrate the efficacy of their programs through accountability structures. Over the years the ownership issue in British Columbia has become increasingly complicated by numerous funding inequities which included the colleges being required to cap enrolments and having to charge tuition. In addition, because colleges fell under the aegis of Ministry of Advanced Education they could not offer the Dogwood credential. The debate over how the two ministries should assume responsibility for their roles as A B E providers began in 1974 and has left a legacy of committees and unresolved recommendations. This legacy included the 1992 Faris Report: Lifelong Learning for the 21st Century: A Report on the Future Development of 33 Adult/Continuing Education in British Columbia. The core recommendations from this report were that: 1. The Ministry of Advanced Education should be designated the locus of responsibility for adult/continuing education. 2. The two ministries would establish a standing interministry coordination committee on adult/continuing education. 3. School districts and college ABE providers would both receive appropriate, equitable levels of funding. 4. All providers would commit to principles of good practice as identified in a province-wide policy development process. 5. A common adult high school graduation credential would be developed. 6. A comparable reporting system on participation and financial data would be developed in colleges and school districts. In 1993 the recommendations of the Faris Report were reviewed by an Interministry Advisory Committee made up of representatives from the University of British Columbia, the college system, and the school districts. The committee concluded that a single ministry should be responsible for ABE, whose minister would regard adult and continuing education as a central, not peripheral focus of the minister's portfolio. The committee further suggested that ABE programs in BC should provide equity with regard to access and fees; there should be appropriate, equitable levels of funding; and there should be a common graduation credential. In June 1994 the government established the MoE/Ministry of Skills, Training and Labour Joint Committee on A B E with the following terms of reference (British Columbia, MAETT, 1996; MAE, 2005). 1. To conduct a review comparing funding and fee structures in the two systems. 2. To make recommendations regarding a consistent policy on fees and funding to the two ministers by December 31, 1994. 34 3. To make recommendations for a common credential for adults for graduation equivalency. 4. To make recommendations for records management, transferability, and certificate granting. 5. To make recommendations about program quality, including program evaluation and guidelines for good practice. 6. To ensure cooperation between the two ministries on issues related to A B E and to act as a forum for resolution of inter-institutional issues. These terms of reference were also the major recommendations of the Faris Report. The ministers requested that a follow-up to the recommendations be completed by December 31, 1994. Coopers and Lybrand was hired in the Fall of 1994 to examine funding for ABE. The firm concluded that the college and school district systems were fundamentally different in their main goals for students: colleges encouraged students to complete prerequisites for further training while school districts required students to complete high school (British Columbia, MAETT, 1996). However, the report's authors found there were enough similarities to warrant further study. They also concluded that once tuition fees for college students were factored into the equation, funding was roughly equivalent across the two systems. They recommended consistency in tuition and student financial assistance. A consultation was held in February 1995 to discuss the report with representatives from both systems. This resulted in continued disagreement with respect to total funding per student and total cost to government per student. Planning an ABE Quality Framework From March 1995 to January 1996, three subcommittees of the Joint Committee were established to develop recommendations for an ABE quality framework, the articulation of 35 courses between the two systems, and a common graduation credential. In September 1995, recommendations on quality, articulation, and the common credential were distributed to colleges and school districts. In December 1995, following a review of feedback, the Joint Committee recommended that the playing field be leveled between the two systems. Specifically, the committee advised that funding to school districts be reduced, that the districts make up the shortfall by charging tuition, that financial assistance be made available to students in both systems, and that a consistent tuition policy be established for both college and school district providers. The committee further proposed that enrolments be capped in the school district system as the college system was already capped and that regional planning be adopted as a cost-saving measure to coordinate the delivery of ABE among school districts and colleges. The committee endorsed a common credential for both systems and advised that an ABE quality framework for program performance and accountability be implemented. In early 1996, the Premier of the province resigned, having taken the "bullet" (as he put it) for NDP insiders misappropriating funds raised by a charity bingo back in the 1970s and 80s. "Bingogate" intensified a growing public demand for accountability in government and in public service sectors such as education where so many tax dollars were being diverted. The political and economic landscape of the province was ripe for the genesis of the A B E framework. By the time the government was ready to act on the recommendations of the Joint Committee, it was the Fall of 1996. The province was then experiencing a period of fiscal restraint, which made accountability an even bigger issue. In order to meet Treasury Board targets for reductions in the cost of ABE to government several proposals were put forward in December 1996 to build alignment between the college and school district systems (British Columbia, MAETT, 1996). The Joint Committee advised that colleges continue to charge tuition for all ABE students except 36 those at the literacy level, while school districts would continue to offer high school graduation tuition free for students up to the age of 21, but charge tuition for students over 21 at a level similar to that charged by colleges, an average of $900 per year for full-time students. Funding for school districts would be reduced for students over the age of 21 and literacy education would be offered tuition free. School district enrolments would be capped at 10% below the 1995/96 level. There would be student assistance in both systems to cover tuition and books. Both college and school district budgets would be cut by 5% as a result of efficiencies anticipated through regional planning exercises. Special category funding would be eliminated in school districts for Aboriginal Education, English as a Second Language, and Career Preparation programs that addressed the needs of adults over 21 years of age. The motivating force driving this flurry of reports and committees was the desire to fine-tune collaborative cross-ministry and performance-based funding strategies as a means to reduce spending on adult basic education (British Columbia, MoE, 1997). It was calculated these initiatives would result in greater savings than the proposals of December 1995. An annual saving of $12.3 million was anticipated. When these proposals were presented to the field, they met considerable resistance from the BC School District Continuing Education Directors' Association (CEDA). CEDA has tended to present an influential medium in the field as it is privy to specialized knowledge. Many CEDA representatives are also members of the BC School Superintendents' Association (BCSSA), attend their meetings, and CEDA appoints a representative every year to work with the BCSSA to ensure adult education has a voice within the superintendent's organization. CEDA's influence and expertise resulted in the appointment of a ministry official specific to ABE 37 programming in 1995: the ministry official attends CEDA's quarterly meetings to ensure that the ministry is aware of field related issues and that the field is aware of ministry issues. Few other professional organizations in the province have access to comparable practical insight and quality intelligence concerning the politics and predicaments of adult basic education. CEDA's use of knowledge and argument has been known to refute and realign the directives of school district superintendents and ministry officials. In the face of CEDA's conciliatory fact-based resistance in this cost-cutting attempt, the government subsequently decided to soften the impact of its proposals significantly. The final government budget for 1997/98 reduced school district adult FTE funding by 5%, capped adult enrolment, and eliminated special category funding for Aboriginal Education, English as a Second Language, and Career Preparation programs. In the Spring of 1998 the Minister of Education, based on an analysis of ministry data, concluded (erroneously, in my estimation) that school district ABE programs had a graduation rate of 8.3%. Based on this analysis the minister announced a cap on enrolment for school districts and a 10% holdback of funds for the following year until a survey was conducted to determine success rates for ABE in the K-12 system. The funding was to be provided in two phases. In the preliminary calculation of grants, 90% of the funding would be included. The remaining funding would be provided in the final recalculation of grants and would be based on school district success rates. To determine school district success rates, a joint committee made up of ministry, district, and adult/continuing education administrators was established in the Spring of 1998. The committee created two key indicators to gather quantitative and qualitative data to aid with decisions addressing the 10% funding holdback. Quantitative data were based on successful completion of ABE courses and qualitative data surfaced personal growth benefits afforded students through 38 their participation in the A B E programs. However, the funding holdback for the 1998/99 school year was based on the quantitative course completion data. The qualitative data were not considered. This lack of appreciation for qualitative data has become one of the most challenging aspects in the design and implementation of an ABE accountability framework. Quantitative results such as that provided by standards-based testing tend to be the preferred rationale for policymakers to base their decisions upon because these measures enable an accountability system to focus on outcomes which provide a convenient means to simplify arguments and give the appearance of equity (Ben Jaafar and Anderson, in press, p. 23). Regardless of intentions to the contrary, the process has a marginalizing influence that reifies learners in the education system whose work and progress need to be understood in nontraditional ways. The Question of Student Success Rates To collect data on school district A B E programs, including student completion rates, the 1998 joint committee of MoE, district, and adult/continuing education administrators conducted a survey. Known as the Educom Survey and administered by Reed and Associates, the data gathered covered the period from July 1, 1997, to June 30, 1998. The survey resulted in the report, Success Indicators for an Accountability Framework for Adult Basic Education in the K-12 System. The request for completion rate data was sent to school districts on June 12, 1998, and responses were collected and returned to Educom by June 30, 1998, a timeframe of 18 days (Reed and Associates, 1998). What Educom found was that when the total number of registrations were compared to the total number of Dogwood certificates awarded in 1997/98 a success rate of 39 21.2% was realized. This exceeded the 8.3% that government had used in determining the need for the short-term study by 12.9%. The survey results also indicated the overall rate of successful course completion in the province was 34.8%. The joint committee believed that given the special circumstances associated with ABE, a better short-term measure of A B E success for funding the 1998/99 school year would be course completion rates, and thus the committee used the overall figure of 34.8%>. However, since no standard for acceptable performance had been established for the K-12 system, the committee was unable to determine whether 34.8% was reasonable or not (Reed and Associates, 1998). A surprising outcome of the survey was that it surfaced documentation of an extremely varied school district A B E course completion rate, ranging from 0% to 86.7%. As a result, three funding options were proposed. 1. Restore half of the withheld 10% funding so that the 1998/99 funding for adults in all districts was 95% of the 1997/98 funding level. Allocate an appropriate amount of the remaining 5% on a sliding scale with a 60% success rate and above receiving the full 5%, and a success rate below 15% receiving 0% of the 5% holdback. 2. Allocate the 10% on a sliding scale, with a 60% success rate and above receiving the full 10%) holdback, and districts with a course rate below 15% receiving 0% of the holdback. 3. Restore the full 10% to all districts pending the completion of a full study. The joint committee recommended the first option, arguing that although it faced limitations with regard to data quality and the short timeline allotted for data accumulation, it would be the most equitable. Given that the ministry had chosen to move to performance-based funding, this option seemed most appropriate as it offered a balanced approach to initiating a performance-based funding model for A B E and provided a one-year transition period to make adjustments in data gathering and performance measures. 40 The ministry used the results of the survey to reduce funding for A B E on the basis of student course completion rates. However, the data from the Educom survey and the Ministry of Education's interpretation of that data were problematic with regard to the MoE's desire to move to performance-based funding for school district A B E providers. The design of the survey was compromised by inconsistencies in the methodology used to gather and report data, limiting its validity, reliability, and generalizability. The survey was conducted in a short timeframe. Some districts had little more than a week to report. The short turnaround period was further complicated by a lack of specific instructions which resulted in general confusion regarding the reporting of course completions. Districts were responsible for reporting course completions. A district's success rate was based on how many course completions it reported. There was no internal mechanism to ensure consistency from district to district regarding the interpretation of instructions on how to report course completions. This created a number of discrepancies. One district, for example, reported a course completion rate of 49.1%, but only reported on 36.9% of its fundable courses. While it reported receiving funding for 941 course registrations and that there were 462 course completions, it was actually funded for 2,551 course registrations. Had it reported its success rate on the basis of total possible fundable course registrations it would have reported a successful course completion rate of 18.1%. Another school district reported a successful course completion rate of approximately 25%. However, because of the absence of clear survey questions and many misunderstandings, it reported receiving funding for 23,384 courses when it was actually funded for 14,875 courses. Had it reported accurately, its successful course completion rate would have been 40.5% (My analysis of data reported in Reed 41 and Associates, 1998). These examples are not isolates. They serve to demonstrate the lack of consistency in district reporting and provide reason to be skeptical of the survey's conclusion that district success rates ranged from 0.0% to 86.7% (Reed and Associates, 1998). Despite the compromised data, the results were still used by the ministry to reduce individual district funding for ABE. The ministry further complicated the intent of the Educom survey by comparing success rates between colleges and school districts. While school district success rates were determined by the results of the Educom survey, college data were based on what colleges defined as success (including many personal development indicators), which did not meet the same criteria as were applied to school districts (these being restricted to course completion rates). Each system, the college and school district, used different processes for gathering and interpreting the data. The baselines for success were internally inconsistent and incommensurable on a system-wide basis. A defensible comparison between the two systems would seem to be difficult to mobilize. Nevertheless, the ministry did the comparison and concluded that school districts had a 21.2% success rate while colleges had a 68% success rate. The comparison created a controversy which did little to assuage the ongoing struggle for ownership of A B E between the two systems. It also raised suspicions of a possible drive to discredit ABE in the school district sector so that some adult programs could be cancelled, others turned over to the colleges, and the presence of ABE programs be reduced as part of a search for strategies to cut government expenses. The Educom survey was based on data that prescribed success on the basis of course completion rates. In this framework, one course registration should result in one course completion, therefore a student registered in eight courses (a fully funded student classified as a full-time 42 equivalent), who attended regularly and worked hard, but completed only two courses in a year, would have a 25% success rate. The college success rates were determined through an accountability process that was based on program contact time not course completion rates. A student who completed two courses in the college system, but attended regularly and worked hard, would be described as completely fulfilling program requirements and therefore be deemed to demonstrate a 100% success rate. A further complication in comparing school district and college success rates was the inequity in funding. Colleges received approximately 35% more FTE funding than school districts (My analysis of A B E College Funding documents submitted to MoE March 10, 2001). This created inequities in the systems with regard to staffing, technology, materials, etc., and had obvious implications when comparing each ministry's A B E success rates. Although the Educom survey failed in its attempt to accurately assess the success of A B E programs, it succeeded in profiling a group of learners who have traditionally been ignored. In this regard the survey may have been a catalyst for better understanding the goals and objectives of ABE and for initiating thoughts on developing relevant success indicators that would best reflect the needs and characteristics of adult learners. It clearly succeeded in signaling to the field the Ministry of Education's resolve to move toward a performance-based funding model that was more accountable to all stakeholders. It also ignited once again the government's enthusiasm for creating a more well coordinated delivery of provincial ABE. This led to numerous initiatives being undertaken over the course of the next few months. 43 Toward a Coordinated System In the Spring of 1998, the Ministry of Education, in collaboration with the Ministry of Advanced Education, Training and Technology, was mandated to develop a coordinated A B E system that was learner centred and provided adult students with maximum access to high quality and flexible educational programs. The proposed coordinated A B E system was intended to afford students the greatest opportunity to meet their educational and career goals in the most cost efficient way possible. To facilitate the mandate of a coordinated A B E system, the MoE and the MAETT established the A B E Transitions Project, which was formed in September 1998 and stood down in August 1999. Conceptually, a coordinated system should be more beneficial and cost-effective than one that is not articulated. However, previous attempts to coordinate school district and college ABE delivery systems were plagued with problems which only served to elevate the degree of divisiveness and dissonance between the two systems. The initial step taken to coordinate the two systems was the May 13, 1998, announcement that college ABE programs would be tuition free for all students regardless of whether they were high school graduates (Andrew Petter, Minister, MAETT, press conference, VCC; Briefing note, MoE/MAETT, February 8, 1999, file no. 280-20). This step, initiated by the Minister of Advanced Education, Training and Technology, was not particularly well received by school district providers of ABE, as students who were high school graduates did not qualify for tuition-free coursework in school district A B E programs. 44 The following month, in August 1998, the Ministry of Education and the Ministry of Advanced Education, Training and Technology seconded individuals from the field to work together to co-chair the A B E Transitions Project and facilitate a coordinated A B E system. The Transitions Project would now be comprised of a more representative body of interests, inclusive of the K-12 and postsecondary systems as well as the MoE and the MAETT. This was my initial involvement in a three-year secondment from my position as the Director of Adult and Continuing Education for School District #79, Cowichan. From August 1998 to August 2001, I was seconded to the Ministry of Education as the Provincial Coordinator for Adult Education. My primary responsibilities were to liaise with the field and the ministry to try and ensure that a more student-centred and accountable delivery system was designed for adult basic education in British Columbia. At first my secondment was to last one year only, but was later extended for two more years. I began my work in the Summer of 1998 to improve the coordination of A B E programs provided by the public postsecondary and school district educational systems. This initiative built on the recommendations of the 1995 Ministry of Education/Ministry of Skills, Training and Labour Joint Committee on ABE. The primary objectives were to establish one adult graduation credential with common requirements; mandate joint college/school district regional planning for delivery of A B E ; and implement a joint quality framework for program performance and accountability. The intended outcomes included improved access and smoother transitions for learners; better articulation of courses between the K-12 and postsecondary systems; more streamlined and faster paths to further training and employment; a labour force with better basic skills, prepared and ready for retraining when necessary; and a more coordinated ABE system. 45 By July 1999, the planning stages for developing the common credential and improving regional collaboration, cooperation, and coordination had been accomplished. Designing the common credential had been my primary responsibility during the first year of secondment. Known as the Adult Dogwood, it now consists of an English 12, a Math 11, two further courses at the grade 12 level, and the option of one additional course at the grade 12 level or Social Studies 11. Regional planning was implemented in the Fall of 1999 with districts required to submit regional planning initiatives in the Spring of 2000. Initial regional planning initiatives were supported through grant incentives provided by the Ministry of Education. The remaining item on the agenda, that of implementing a common accountability framework for both the school district and college sectors, proved more elusive. In Pursuit of a Common Accountability Framework In the Spring of 1999 a Transitions Project subcommittee consisting of representatives from both ministries and both systems was established to develop a common framework for quality and accountability. Based on our academic and field experience, members of the cross-ministry accountability subcommittee determined that the goals and objectives of adult education were to provide access to preparatory courses and skills ranging from basic literacy to provincial adult secondary coursework which would lead to further education, employability skills, and life management competencies. Indicators of success could include course completions, postsecondary admission, employment, academic skills, technology skills, attitudinal and responsibility changes, adaptability, time management, confidence, collaborative behaviour, decision-making ability, etc. 46 The Transitions Project subcommittee concluded its work on identifying ABE's aforementioned goals and objectives, but difficulties remained. Considerable confusion existed as to how an accountability framework for assessment of such a multidimensional understanding of the purpose of adult basic education could gain political support. Qualitative dimensions of program success are often thought of as "happiness indicators", and although funding bodies pay lip-service to their value, they are seldom included as legitimate success indicators in funding formulas for public educational institutions. Furthermore, while the intent of the 1998 ABE Transitions Project was to design a standard accountability framework for colleges and school districts, the reality of a cross-ministry accountability framework was confounded by a number of system inequities. First, there was the inequity of tuition fees charged to A B E students in the different systems. As discussed, colleges had gained tuition-free status for all A B E students including high school graduates. High school graduates enrolled in ABE programs as a means to upgrade their academic qualifications and further their employment or education opportunities. However, high school graduates in school districts with the very same motivations as those in the college system could not access tuition-free adult basic education. Second, there was the inequity of government funding provided to the two systems. Colleges received approximately $2,000 (plus or minus) more per FTE than school districts. Instructional costs were similar, approximately $4,000 per FTE. However, colleges received additional monies under the general support allocation of their program profile, which included funding for facilities, materials, etc. Few, if any, school district adult learning centres were housed in existing elementary or high school structures and their general support costs were as high, if not 47 higher than colleges, as many districts leased space for their adult learning centres. The FTE funding difference between school districts and colleges negatively impacted the financial support for school districts to enhance accountability and performance. Third, there was the inequity of the Adult Basic Education Student Assistance Program (ABESAP) available to students in the two systems. ABESAP was only available to college students. Like college students, many school district adult students were not able to afford related A B E costs such as books, materials, and supplies. Eventually, some intersectoral inequities would be addressed, but not in ways supportive of the needs of ABE students.4 In 1999, however, none of these inequities seemed likely to be ameliorated in the near future. The efforts of this initial cross-ministry enterprise thus culminated in the design and implementation of the common credential and regional planning for the Ministry of Education's A B E delivery system. The subcommittee of the A B E Transitions Project concluded that the funding inequities between the two systems precluded an accountability process common to both the MoE and the MAETT. The goal of creating a coordinated ABE system that would integrate the college and school district sectors had reached an impasse. With a common adult graduation credential in place, regional planning and course articulation between the two systems advanced further toward creating a more harmonious system focused on the delivery of learner-centred, high quality, flexible programming. But the proposal for a common framework posed deep-seated challenges embedded in systemic funding differences that government did not ameliorate and school districts would not buy into. Although there was never a collaborative decision made to resolve the issue, the seconded school district director was held over for another two years. The MoE's focus was now on the design of an 48 accountability framework suitable to the province's school districts, the development of which is addressed in the remainder of this dissertation. Summary This chapter traced the development of government policy dealing with adult basic education in British Columbia from the 1960s to the decision to design an A B E framework specific to the needs of school districts. It noted that A B E was briefly a federal interest when the Canadian government's state-building ambitions focused on ways to integrate a multicultural population upon which the country's future prosperity was based. Subsequently, during the economic recession of the early 1980s, responsibility for funding A B E programming fell back to the provinces where it had long been, a move that was rationalized by the constitutional circumstance of education at all levels being a mandate of the provincial governments. Jurisdictional differences have since been the hallmark of A B E programming in British Columbia. Whether federal or provincial, or between the Ministries of Education and Advanced Education, or among school districts and colleges, jurisdictional inequities - which tend to be inscribed by other power differentials such as funding inequities - have delayed the development of a highly coordinated system. School district and college A B E providers have the same agenda to provide A B E to a fairly heterogeneous audience but must do so using different curricula directed toward different ends (grade 12 graduation vs\ personal growth) under different funding conditions that are contingent on different measures of success (course completion rate vs. personal transformation). Despite these incommensurabilities, there was sufficient support among the ministries and the sectors for 49 significant forms of intersectoral coordination to be accomplished. The Adult Dogwood was established as the common adult graduation credential in 1999 after a year of extensive and intensive negotiations among the sectors involved (and after more than two decades of attempts to institute the credential). A system of regional coordination was also implemented in the same year. Designing a common A B E accountability framework for school districts and colleges was a more complicated matter, confounded from the start by system inequities in tuition fees, government funding, capital costs, instructional expenses, and access to student loans - all of which tended to favour A B E in the college system. Nevertheless, the collaborative brainstorming and collective expertise nurtured in these intersectoral endeavours in addition to the research and conceptual studies I was charged with bringing to the design of a common framework would eventually lay the basis for the development of an accountability framework unique to the school district sector. The concepts that influenced the framework's design and subsequent analysis of its efficacy are discussed in the next chapter. Following that, I describe the events and impediments the MoE and school district representatives faced in constructing and implementing an A B E accountability framework applicable to British Columbia's K-12 structure. 50 Notes 11 am very grateful to External Examiner Dr. Nicholas Rubidge for this historical information. My review of A B E teacher qualifications in the college sector examined employment postings on the institutions' websites and was substantiated through conversation with A B E instructors in the college sector and an official with the Federation of Post-Secondary Educators of BC (FPSE), a union representing instructional staff in the colleges and university-colleges. 3 The absence of training requirements for ABE instructors has long been a significant issue. In 1983 Audrey Thomas wrote "Adult Illiteracy in Canada - A Challenge" for the Canadian Commission for Unesco (Thomas, 1983). Speaking of the professional qualifications of ABE teachers in Canada generally, Thomas observed In institutional ABE/literacy classes, instructional staff tend to have the following profile: white, married, female (about 80 per cent), working mainly on short-term contracts or a part-time basis, no previous background in A B E or adult education training. Most 'fell into A B E ' because they needed a job and something became available locally. Previous training and experience has usually been in the lower levels of the public school system, special education, or perhaps in some social work type of activity. (1983, p. 110) The Thomas report argued the "little formal training offered in A B E " was a consequence of "a lack of commitment to A B E at the university level", with the University of British Columbia being the only university at the time in Canada that was "apparently making serious attempts to construct and deliver training opportunities in A B E " (pp. 109, 110). This lack of professional standards and status is not unique to Canada (see NCSALL, 2003, pp. 5-10, for a recent critical overview of qualifications of ABE teachers in the US). The Occupational Outlook Handbook of the US Bureau of Labor Statistics (2006-2007 edition) similarly reports that a bachelor's degree or teaching certificate is all that is required to teach A B E in most states and that such work often relies on unpaid volunteers with paid staff generally being uncredentialed part-time instructors employed without benefits (US Department of Labor; see also Perrin, 2003). In 2000 one of the more educationally progressive states in the union was only beginning a process "to develop a certificate that would be available to, but not required of, teachers of adult basic education" (Massachusetts Department of Education, May 2000). 4 In 2002 the Liberal government made several changes to college A B E funding, including the reintroduction of tuition for ABE students and the elimination of ABESAP. At the district level the government introduced a new funding formula for A B E that significantly impacted school district delivery of A B E and restricted access to students who were most likely to be successful, further marginalizing those already marginalized. Within both sectors, A B E has come under policy changes that do little to overcome illiteracy in BC, but instead promote the dependency of undereducated adults on a state that is downsizing, privatizing, or eliminating its social services. The policy changes A B E is currently adapting to will be further discussed in the conclusion. 51 Chapter Three Conceptual Sensitizers Educational accountability is a problematic concept, or juxtaposition of concepts, which presupposes a number of contradictory meanings and motives that may tend to build resistance among educators dealing with the term. Resolving the ambiguous nature of educational accountability is contingent on better understanding the multiple purposes of adult education and accountability, and working toward a more effective marriage of the two concepts. The research literature examined in this chapter is foundational to bringing about this alignment and is requisite to the development and analysis of the A B E accountability framework reviewed in this study. The theory and research surveyed pertain to the educational accountability of adult basic education (ABE) programs offered through K-12 systems. The focus is on the fields of adult education, adult basic education, and educational accountability. Within these domains, 1 explore the role of ABE, the debate over approaches to educational accountability, the complexities involved in determining accountability for adult basic education programs, and some strategies to encourage a collaborative effort to effectively put into practice an all-embracing understanding of the concept. Adult Education The field of adult learning has origins dating to at least 1920, when Lindeman introduced the term "andragogy" to distinguish adult learning from pedagogy, or the teaching of children (Nixon-Ponder, 1995). Since the 1950s, adult education has emerged as a unique field of educational study characterized by considerable debate about which educational activities are 52 most effective for adult learners and how these can best be developed and implemented. From the 1970s, when Knowles (1980) added theoretical substance to the term andragogy, research in adult education has focused on the need to better understand the multifaceted differences that distinguished adult learners from children. At the level of structure, one of the main differences can be found in terms of institutional purpose. Adult educationalists have long suggested that the purpose of elementary and secondary education is more easily understood than adult education (e.g., Darkenwald and Merriam, 1982; Knowles, 1980). In this view, schools in the K-12 system are seen to function as agencies of socialization that inculcate children and adolescents with the values of the dominant culture (Darkenwald and Merriam, 1982). These values reflect middle-class life styles and serve to reproduce the prevailing economic structures (Quigley, 1991). The primary mandate of schools in these systems is to prepare students for a credential by using curriculum predicated on sociopolitical standards that provide students with pre-packaged skill-sets designed to enable them to contribute in some way to the economy (Delandshere, 2001; Grundy, 1992; Quigley, 1991). Achievement within these curriculum standards is most often measured through test-driven assessment. Adult education, on the other hand, is not so clearly defined. In its broadest sense it encompasses practically all experiences through which mature individuals learn and acquire knowledge (Knowles, 1980, p. 25; Cross, 1981, p. 52). Rather than socialize a future workforce, it is most concerned with helping "people to live more successfully" (Darkenwald and Merriam, 1982, p. 9). In this sense the function of adult education may be increasing competence in social roles, finding greater fulfillment in personal lives, or improving capacity to solve personal and 53 community problems. In a more technical sense adult education signifies a set of organized, formal or informal learning activities sponsored by a number of institutions and aimed at accomplishing specific educational objectives which may or may not include learning for credit (Knowles, 1980, p. 25; Cross, 1981, p. 52). It may refer to specific self-directed learning practices which although organized by others are usually planned and directed by the learner (Cross, 1981, p. 52), or it may refer to a general field of social practice concerned with the education of adults (Knowles, 1980, p. 25). Selman and Dampier (1991) surfaced a difference between "the education of adults" and "adult education" per se: whereas the education of adults refers to "all purposeful efforts by which adults seek to learn, or are assisted to learn", adult education "tends to refer to a narrower spectrum of such activities . . . designed especially for adults", which are usually "part-time or, if full-time, of relatively short duration" (p. 2). A further goal of adult education is emancipatory learning which emphasizes learner transformation through processes of enlightenment, understanding, and self-intervention. Emancipatory learning, according to Mezirow (1991), can only take place in adulthood as it is only then that a person can recognize being caught up in their own history and reliving it. The features that define adult education also apply to adult basic education; however, ABE is somewhat more complex. At the simplest level, ABE can be understood as educational programs for students who dropped out of school and are now adults or who needed to upgrade their qualifications. At a more complicated level, A B E is an educational activity consisting of a diversity of program designs and delivery formats which may be offered full-time or part-time, at the elementary and/or secondary education levels, with goals and objectives that may be multiple, and in some situations its outcomes may be more emancipatory than academic. ABE in this study is further confounded by the K-12 mandate, which superimposes assessment measures 54 designed for children onto an educational activity that enrolls adult students, most of whom had experienced little initial success as children within those K-12 structures. To begin to grasp the design requirements of an effective accountability framework for measuring success among adult basic education programs and students requires that the characteristics of these learners as distinct from the general population of adult and child learners be more clearly understood. ABE Students Who ABE students are is a critical aspect in constructing an A B E accountability framework. If we knew these students were all highly motivated, academically capable adults who were now ready and able to fulfill their graduation requirements then it would be fairly straightforward to design an accountability framework. A standardized approach under these conditions would be effective. However, this characterization of ABE students does not accurately depict the majority. While there are some highly motivated, academically capable adult basic learners who are well prepared to fulfill their graduation requirements or upgrade their qualifications, this descriptor does not have universal or even wide application. Mezirow, Darkenwald, and Knox (1975), at the close of two years of research on urban ABE programs in six American cities, concluded that "urban A B E students are likely to be young to middle-aged, poor, and employed as unskilled or semi-skilled laborers or service workers" (p. 39). They found that about 60% of those enrolled in A B E programs were women and "although generally employed, the undereducated adults in A B E labor in low-status, low-paid, hard, and generally undesirable jobs" (p. 41). Some fifteen years later, Beder and Valentine's (1990) study of a sample of 3,090 A B E students in Iowa reported similar findings for male and 55 female learners living in rural and urban areas. Of this sample, those considered "highly motivated" comprised about 18.6%. A substantial proportion of young adults disengage from traditional educational pathways and resurface in ABE programs. In British Columbia alone, a critical examination of records from the Ministry of Education's data collection branch indicated a longitudinal grade 12 graduation rate of approximately 70% for school-age students (My analysis of data reported in British Columbia, Ministry of Education, 2001). This rate that has remained virtually unchanged for the last thirty years or since the tracking of dropout rates began. Reasons for initial dropout are complex; however, most students will trace a significant part of their disenchantment to unsatisfactory and often unhappy school experiences. They were unable to experience success within traditional K-12 expectations, and their frustration within that system frequently culminated in truancy and exclusion (Boudreau, 1998). Boudreau's (1998) study of ABE programs offered through BC's public school districts found that many students indicated they had left the traditional school setting for a variety of reasons that included school culture and rigidity in academics and structure. They saw being immersed in a traditional teacher-directed school environment as negating their desire to perform. As adults, they were attracted to a nontraditional school environment because of its apparent lack of formality and more humanitarian philosophy with regard to learning. Many A B E students are likely the result of being misfits in the hegemonic success structures associated with traditional K-12 systems which tend to favour students from socioeconomic backgrounds closely aligned with the predominant cultures (Grundy, 1992). Valencia (1997) described the "one size fits all" interpretation of success as a form of social thought that does not 56 recognize learner difference. For many ABE students, this was likely a catalyst in their initial disengagement from schooling. The influence of K-12 structures may result in what Valencia (1997) described as "deficit thinking". As a theory of school failure, deficit thinking bases failure on internal deficits or deficiencies and is often described through the "at-risk" construct that is now well entrenched in educational circles. Founded on imputation and not documentation, deficit thinking ascribes failure characteristics such as limited intellectual abilities, linguistic shortcomings, lack of motivation, and immoral behavior, to genetics, culture, class, and familial socialization (Valencia, 1997). This social construct of failure, which attributes failure to the student, is often linked to socially constructed terms such as "culturally disadvantaged" and "culturally deprived" (Valencia, 1997). Scheurich (1994) argued that the process of labeling targeted groups, especially when legitimated by social agents such as professionals, disciplines citizens by reaffirming their opinions of what is and is not good, worthwhile, and proper. These attitudes are not only held by mainstream "productive" citizens but also impact the self-concept of those on the outside looking in. The role of education in combating social exclusion has become a point of debate in international policy circles (e.g., OECD, 2000). Social exclusion may be understood as "the inability to participate effectively in economic, social, and cultural life and, in some cases, alienation and distance from mainstream society" (Duffy, 1995, cited in Klasen, 2000, p. 2). Its primary cause is a "lack of access to the institutions and facilities of learning" (OECD, 2000, p. 109). It is considered a major contributing factor to many forms of anti-social and criminal behaviours (e.g., Young, 1999). Educational policy can either promote or stem the development of social marginalism. Internationally, countries are concerned with raising test-based standards of educational achievement in an effort to become more competitive in a world economy. This 57 may (and in B C often does) promote a situation where "the best" students - that is, not A B E students but those most able to work within conventional assessment structures - receive disproportionately more resources than below average students to pursue their educational endeavours. This may also lead to admission requirements being adjusted so that only students deemed to be highly motivated are accepted into A B E programs. Both processes, known as "creaming" among education practitioners, act as marginal izing influences that exclude those who are most in need o f educational guidance and most "at r isk" (Klasen, 2000). The majority o f educationally disengaged adults report that their disaffection arises and continues as a result o f a complex interaction with structural factors which sociologists refer to as "agency" and "structure" ( N I A C E , 2000, p. 2). The structural factors comprise an education and employment system in which the students and their families have not thrived, and in response, they may have chosen to instigate attitudes and behaviours that put themselves at even greater distance from it. Their past experience with the education system is an important indicator o f marginalization. Those who fail to achieve at school are more l ikely to find themselves on the margins o f opportunities and o f the communities in wh ich they live. Most school leavers have low levels o f self-esteem, motivation, and expectations for the future ( N I A C E , 2000; Statistics Canada, 2001). Adults attracted to A B E programs often feel embarrassed and inadequate. Their past experience with schooling was humiliat ing. They felt like failures then, and now as adults returning to school many feel ignorant. They often have a history o f marginalization and social exclusion. They not only tend to be financially poor, but may also be from social groups whose ethnicity, culture, and identity mean little in the way o f recognition, influence, and power in Canadian society. They are l ikely to be the least wel l 58 educated and the most disenfranchised when it comes to active participation in civil and democratic forums (Robinson, 1994; Marshall, 1997; Gaskill, 1988). Many enroll in ABE with the understanding that there will be greater flexibility in system expectations regarding progress and attendance. Many will be disappointed, and consequently will leave school again. Kerka's (1995) examination of retention rates of A B E students in the US observed that attrition is reported to be the number one problem. She noted that rates "as high as 60-70% are reported in state and federal statistics" (p. 1). Using Quigley's (1995) data on A B E attrition, Kerka observed that many A B E students even though they are motivated to enroll in educational programs dropped out because their negative past experiences with school resurfaced when they encountered classrooms, teachers, curricula, and structures reminiscent of those past experiences (p. 2). Those who stay, however, may experience many effects that go beyond the ability to successfully demonstrate the acquisition of course content. Research indicates that the outcomes of ABE programs are multiple and reflect more than course acquisition. Beder and Valentine (1990) conducted research to better understand adult motivation to participate in A B E and the outcomes of participation. They concluded that acquisition of simple literacy skills resulted in far-reaching personal and social outcomes which enabled participants to rid themselves of internalized feelings of inadequacy and prepared them to make further changes in their lives. Likewise, Boudreau (1998) researched adult basic education learning centres in the public school system of British Columbia and found that "self-esteem is the first acquisition of the adult students once the initial fear of being back in school has diminished" (p. 41). Personal growth is a highly significant though relatively common outcome of participation in A B E programs. Furthermore, as Cross (1981) once hypothesized, enhanced self-confidence appears to contribute to success in adult learning as well as to success 59 in life. This was substantiated in Upex's (1999) study, where 122 adults enrolled in A B E programs offered through six adult learning centres in British Columbia were interviewed to examine what the students believed sustained them in their efforts as learners. The study revealed, among other factors, that self-confidence and self-knowledge were the gains students valued most. These attributes may lead to further developments. As Ranson (2000) observed, literacy is "more than an instrumental skill", it is "the core communicative capability which empowers people to express their voice as citizens in the community" (p. 36). In his view, not only is self-esteem of vital significance to the development of "voice", but the ability to articulate is a critical skill-building attribute. In Ranson's view, voice is essential to developing "intelligence" which he contends is not a fixed internal characteristic but is learned through experience, diligence, social interaction, and developing capacities for critical self-reflection. Students who drop out are excluded from the intrinsic and instrumental benefits offered to those who successfully graduate from secondary school. If they return as A B E learners, they may face further stigmatizing evaluation and support structures. Structures that rely on standardized testing do not allow for individual difference. They too often focus on academics only and do not address the social and emotional needs of many ABE students. This over reliance on academic indicators does not recognize or value non-academic indicators of student success such as self-esteem, motivation, and wellbeing (Klasen, 2000). These accountability structures inadvertently promote further social exclusion and marginalization and are counterproductive to both the emancipatory and reproductive purposes of education. As Bynner et al. (2000) suggested, the design of an effective accountability framework for disaffected learners would seem to need to recognize the academic, social, and psychological diversity of the people attracted to the programs and the value of learning outcomes that are not only academic and systems driven. The 60 following discussion addresses theories of accountability that offer insights into the design of accountability structures that are more well considered and inclusive. They reflect the K-12 system as that structure is what school district A B E programs function within and is the environment that contributed to the initial disenfranchisement of early school leavers. Educational Accountability The development of an accountability framework for ABE is a policy initiative that provides politicians with an argument to support or terminate publicly funded A B E programs. Prudence dictates that any argument for a nontraditional accountability framework will garner greater political support if it is guided through conventional accountability frameworks. Two particularly thoughtful and coherent traditional approaches to accountability are provided by Boyle (1981) and Boone (1985). Boyle (1981) sets the stage for understanding what educational evaluation is: a judgement cast upon the worth or value of a program which is formed by comparing evidence of what a program is against criteria of what it "ought" to be. The evaluation process involves establishing standards and criteria, gathering evidence about the criteria, and making judgements about what the comparison reveals (ibid., p. 225). Criteria consist of measures forjudging something. They describe what constitutes a valuable program, identify what needs to be known about one currently in operation, and direct the search for particular kinds of evidence. Evidence is compared to the criteria to determine program effectiveness. When the objectives of the program are stated in terms of change, evidence of program effectiveness can reflect shifts in income, physical changes, socio-educational changes, or changes in the cultural environment. 61 Characteristics to evaluate in adult education programs include quality of activity, curriculum content, and teacher performance, which are measured in terms of their suitability at meeting learner expectations, effectiveness at accomplishing objectives, efficiency with regard to use of resources, and importance for those who participated (p. 228). Although often overlooked when translated into practice, this conventional approach substantiates the need for diverse sources of evidence, as Boyle's mode of evaluation is based on hard data such as quantitative outcomes compiled through test results and completion rates, as well as soft data, which are qualitative in nature and may include perceptions of changes in attitudes and aspirations. Another conventional approach is Boone's (1985) accountability system which is based on the assumption that the primary purpose of adult education programs is to affect "behavioural changes in a specified public" (p. 171). In this model accountability assessments consist of measuring inputs, assessing outcomes, and using the evaluation data to restructure programs (p. 171). Boone suggests that outcomes may manifest as intended or unintended changes of behaviour which may be characterized in four ways: (1) intended outcomes which are demonstrated through the achievement of specified objectives; (2) unintended or incidental outcomes which are also important in program evaluation; outcomes that are (3) manifest and observable immediately, or (4) latent and observable some time after the learning activity has been completed. Thus Boone supports the need for a diversity of measures to gauge student and program success since the unintended and latent outcomes of adult basic education such as changes in self-esteem are not tied to academics and are not discernable through standardized test results. 62 In addition to incorporating some conventions from these models an accountability framework seeking to legitimize the use of nontraditional indicators of success would address the economic, social, and technical arguments that routinely frame the rationale for implementing accountability processes in education. These three domains shape our understanding of educational accountability and clarify its purpose. They are foundational to the controversial nature of educational accountability and highlight a number of issues significant to the analysis of an ABE accountability process. Economic Accountability "Whatever else accountability may be, it is a way of holding down spending", argued House (1972) in an earlier accountability era (p. 13). Accountability was born of economic problems (Glass, 1972). It emerged with the worldwide economic recession of the late 1960s and early 1970s, and continues to be driven, Cibulka (1990) points out, by two arguments predominant in the political economy of our times. The first is that "the public", meaning the taxpayer, has a right to know how schools are performing. The second is that countries competing in a global "knowledge economy" are compelled to exhibit improved performance in a range of cognitive skills (p. 182). A principal economic rationale for educational accountability is the argument that standards prepare students for the labour force (Levin, 1998). Accountability, in this view, is based on the assumption that the "purpose of education is to supply manpower to other institutions of society, particularly the economic one" (House, 1972, p. 15). Being accountable means maximizing known objectives through assessment and control mechanisms applied to learners, teachers, and the schooling processes. Students (and their teachers) are shaped to prespecified ends in 63 institutions functioning as unchanging and unchangeable objective realities that demand an individual's accountability but in turn resist being responsive to the needs and differences of the students in attendance (ibid.). In an argument compatible with House's industrial model of mass education, Glass (1972) suggested that educational accountability is an attempt to rationalize the relationship between students and the educational system. The objective, as he saw it, is to represent the relationship as a simple economic transaction between vendor and buyer (p. 636). Educational accountability is thus an indication of the advancement of the industrialization of education which introduces a new set of culpabilities that institutions are required to address. For an accountable relationship to function effectively in this perspective, Glass suggests three keystones need to be set in place: 1) disclosure with regard to the product or service being sold; 2) product testing to provide some assurance of future reliable performances; and 3) a means of "redress in the event of false disclosure or poor performance" (p. 636). The rationale for educational accountability reporting is not entirely clear, however, and the political motivations for gathering performance information are questionable to many (Cibulka, 1990, p. 181). When Levin (1998) reviewed the predictability of performance assessment standards in the US to determine the economic productivity or quality of the workforce, he concluded there was no evidence to link performance standards to economic productivity (p. 8). As he and Kelley observed in an earlier study, test skills do not correlate to team skills and work habits that are important aspects of organizational performance (Levin and Kelley, 1994). Levin nevertheless continued to support new approaches to accountability so long as they reflected the 64 "contexts and situations that students and adults face" in the working world, which, he observed, go beyond test scores and may lead to a more realistic predictor of economic productivity (p. 7). Levin's (1998) argument is reminiscent of Macpherson (1996) who similarly argued that there was "no convincing causal story" for linking the stated need for increased educational accountability with policies that tended to measure "inputs, processes, and outputs" in standardized ways (p. 90). There was more evidence, in his estimation, to suggest that such strategies not only deskilled teachers (because of tighter curriculum controls) but also favoured educational institutions in districts that were the most economically advantaged. The impact of such policies, he submitted, was that the term "accountability" fell "into disrepute among educators. It tended to be accorded meanings in mercantile, industrial, or political metaphors rather than be related to notions of morally responsible schooling" (p. 90). Macpherson posited a "systems" approach to educational accountability that honored and was inclusive of the many understandings of accountability held by public education's various stakeholders. The systems approach will be examined further in this chapter in the section entitled Designing Accountability Frameworks. Social Accountability Assessment practices are a means of shaping intellectual and moral activities to meet the requirements of society's dominant power structures. The notion that assessment mechanisms contribute to the organization and reproduction of the social and intellectual order was explored by Delandshere (2001), who studied educational assessment as a sociopolitical practice (p. 118). He examined Bourdieu's (1989) analysis of schools in France which documented how "school assessment processes appear to reproduce unconsciously the values, and form of appreciation 65 inherent to the structure of the distribution of economic and cultural capital" (Delandshere, 2001, p. 121). While discussing test content, he noted that tests support a dominant "decontextualized" knowledge, meaning a knowledge reflective of white, middle-class contexts and marginalizing of others. Such "objectified" knowledge requires compliance and "sets in motion the hierarchical relations of power and authority that exist in the larger society" (p. 128). Through systems of examination, judgement, selection, and certification of competence, the role of knowledge in support of the dominant order is reinforced. Decontextualization creates an enigma with regard to schools whose test scores may be below average, as it questions whose average they are failing to achieve. For Delandshere, standardized testing has the ability to further separate the school's priorities from its community, as stakeholders do not have an opportunity to dispute educational assumptions, but must merely comply. The unnoticed structures that shape our perceptions of social reality are conceived to be "social regularities" in Scheurich's (1994) Foucaultian approach to educational policy analysis. These regularities, or "rules of formation", configure the social, moral, legal, political, and economic guidelines that frame permissible and even conceivable thoughts and actions. In doing so, these grids of social regularities occlude some ideas, knowledges, and world views while perpetuating the ordering rationale embedded in others. The "common understandings" they reproduce designate what becomes "socially visible as a social problem" and constitute "what becomes socially visible as a range of credible policy solutions" (p. 301). Social regularities thus have the potential to orient and box in a group's way of thinking which may bias and impede the development of an accountability process. 66 Of interest to the development of ari ABE accountability framework is Scheurich's (1994) discussion of professionalization and governmentality. Professionalization is a social regularity that is manifested through the training of professionals, in this case K-12 teachers, who are professionally socialized with regard to the role of schools in reproducing the social and intellectual order. This is not only accomplished through indoctrination within the teaching profession but is also maintained and entrenched through politically driven assessment structures that prescribe what is worthwhile in education (p. 307). Governmental ity as a social regularity is a form of governance rationality that reinforces the social order through the management of zones of social life. The proliferation of this social regularity is managed through government agents who are likely not cognizant of their involvement in the perpetuation of behaviours that predetermine the wellbeing, happiness, and productivity of individuals (p. 306). The replication of the social order through governmentality and professionalization facilitates the continuity of entrenched power inequalities by creating and maintaining a marginalized social group that is the antithesis of a happy, healthy, and productive citizenry. By labeling some social groups "at risk", those most marginalized by prevalent social structures serve as icons of failure and exclusion, reaffirming the need to conform and subscribe to the regularities that are foundational to supporting society's network of institutions, as is the K-12 educational system. Technical Aspects of Accountability Educational assessments are increasingly made on the basis of judgements formed with the assistance of statistical measurement instruments that are increasingly more refined, codified, and standardized. Through quantification instruments, psychometricians, evaluators, and other measurement experts offer statistical interpretations of learning and understanding. By providing 67 the legitimacy of scientifically validated knowledge, the technology of assessment, and particularly the use of educational testing, has become a commonplace policy solution for policymakers dissatisfied with the quality of education. Education researchers tend to be skeptical of the technical approach to measuring accountability. Assessment technology may point to a problem but is not a remedy and actually does little to "inform and support change in teaching and learning" (Delandshere, 2001, p. 115). It tends instead to result in the reification of learners and the commodification of educational programs (Grundy, 1992). Standards and measures often fail to value the social nature of education and the context in which educational programs are offered (Grundy, 1992; Macpherson, 1996). This has been the case in British Columbia where a government-sponsored study indicated that initial approaches to ABE accountability focused only on course completion rates, and failed to recognize other indicators of success relevant to the goals and objectives of ABE (Moss, 2001). Performance indicators that are defined through the quantification of educational objectives encourage a technical orientation towards practice, where teachers and learners are engaged in a form of interaction designed to bring about pre-specified ends (Grundy, 1992). This has significant implications for the purpose of education and especially the development of an accountability process for ABE, which does not lend itself well to pre-specified ends. Thompson (2001) points to the "high-stakes, standardized, test-based reform" mode of standards-driven assessment tools as one which undermines the building of effective and inclusive education systems (p. 358). Academic performance is in this case judged by a single indicator, and is "high-stakes" in the sense that whether a student is promoted or eligible for a diploma is 68 determined by that one indicator. This approach to educational accountability, ThompSl observes, reduces instruction to test preparation. Sociopolitical Implications The economic, social, and technical domains of educational accountability discussed above bring to the fore a number of consequences for districts, schools, and students when accountability measures fail to adequately account for individual differences. System-wide approaches to accountability can be particularly blunt instruments. They tend not to reflect social consensus, but are often comprised of simplistic across-the-board strategies (House, 1972). Rather than encourage a responsibly accountable school or district, systems approaches are not only limited in their ability to explain or control changes in practice but provide persistent evidence of performance measurement corrupting rather improving schooling, as it becomes all too easy to manipulate and misuse the results (Macpherson, 1996, following Griffith, 1990). Systems approaches may not allow fair comparisons between schools and students when students begin at different levels, with varying resources, and add value in different ways in a context where the purposes of education and the rationales for making comparisons are disputed. The current politically-driven movement for standardized high-stakes, test-based assessment to determine school performance is in response to demands for rapid, quantifiable, and easily digestible data (Thompson, 2001). Consequently, educational accountability has adopted a business management orientation. This encourages the responsibility for test results to be downloaded to schools, where teachers and administrators become the handmaidens to a decontextualized definition of instruction and school performance (Thompson, 2001). Student performance becomes the responsibility of individual schools (Pipho, 1989), which are 69 increasing singly likened to businesses. Through this process only the fittest survive, and, like any other commercial enterprise, if they cannot produce effective results on tests they face downsizing, rationalization, and lay-offs (Eisner, 2000). Test-driven accountability structures mean that teachers are not only held responsible for student test scores but that teacher salaries and promotions could weigh in the balance (Knowles and Knowles, 2001; Pipho 1989). This form of accountability encourages test-driven teaching which tends to result in a narrowing of the curriculum and inattention to learner differences. This inevitably leads to more students becoming "leftovers" (Knowles and Knowles, p. 391; see also Macpherson, 1996, p. 88). Eisner (2000) believes that as a performance measurement practice, high-stakes standardized testing represents a determined effort to rationalize the educational process. This rationalization of education depends on specifying intended outcomes through the use of standards that proclaim sociopolitical values such as cost efficiency and economic competitiveness (p. 368). Rationalization is predicated on the ability to predict and control, Eisner points out, and measurement through quantification becomes the mechanism to assure quality control. Rationalization imposes a commensurability that cannot reflect learner differences or the varying socioeconomic and demographic differences between schools. The purpose of education is effectively, and some might argue deliberately (e.g., Fraser, 1997), changed, since instead of enabling many students to do better in life the rationalization of education only enables a small proportion of them to do better on tests. Test-based assessments promote a system of resource allocation that leads to the provision of the highest quality education to the best and the brightest (Pacquette, 1998), and ultimately to the construction of further social inequality. An ABE accountability process that does not address 70 the educational abilities and needs of those who have difficulty demonstrating learning gains through traditional K-12 assessment structures will continue to benefit those with the highest levels of educational attainment and marginalize those with the lowest levels (Wrigley, 1998). The effects of this are cumulative in a "knowledge society", making it increasingly difficult for those most marginalized to break out of the social regularities that organize their world. This type of marginalizing is alive and well in British Columbia's K-12 system. Apprenticeship and career programs that emphasize experiential learning are often a preferred learning style for at-risk students (Sticht et al., 1987). As the students enrolled in these programs are funded on the basis of traditional test-driven measures, those who are the highest achievers have priority access to such programs while those defined as at-risk within the success structures of the regular K-12 system are least likely to access apprenticeship and career programs, which further restricts their ability to succeed in life (Grundy, 1992; Valencia, 1997). The course completion approach to measuring accountability is another policy instrument that serves to marginalize learners. In this approach, as practiced by the MoE's programs in ABE, each district's success rate is dependent on the number of courses it is funded for and the number of courses successfully completed; that is to say, the number of courses completed, not the number of students attending, determines each district's FTE values and MoE funding. As indicators of student and program success, course completion rates are determined through standardized structures that are test-driven, tend to favour the more affluent school districts, and do not reflect other important outcomes of ABE. The course completion mode of program and student assessment fails to grasp the context of ABE and the diverse nature of adult learners. It is also at odds with the concept of lifelong learning which at the time of this bounded study was 71 upheld by the MoE as one of its mandate's chief objectives (British Columbia, Ministry of Education, 1999a). The Ministry of Education's perspective was informed by Faris's (1992) report on the development of Adult/Continuing Education in British Columbia. The Faris report advised that the provincial government Adopt the concept of lifelong learning as an organizing principle and social goal for education and training programs of all government ministries and agencies, the total public education system, and the promotion of learning opportunities in the non-formal sector. . . . (p. 14) At that time, the provincial government appeared to be concerned with the prospect of building a "learning society". To do so would require a policy intervention that made special consideration for marginalized adults, as it is not only they who are at risk but generations of young people under their influence. In an increasingly heterogeneous society, this impacts the social cohesion, political stability, and economic development probabilities of society (OECD, 2000, p. 109). Education is an attribute of familial culture and social class that is habit forming. As Rubenson (1997) pointed out, the most educated people in terms of time invested in formal systems are the most likely to participate in further education and lifelong learning. Recent research on Canadian adult education reaffirmed that those with the lowest levels of education were the least likely to participate in further education (Statistics Canada, 2001). The lack of education results in the exclusion of certain groups from fully engaging in Canada's expressed ambitions to become a "knowledge society" (Ranson, 2000; Statistics Canada, 2001). 72 There is the possibility that in the eyes of some policymakers a degree of social exclusion is an entirely serviceable state of affairs. The lack of support for programs that are not measured by standardized success indicators may be symptomatic of the realpolitik occupying educational policymakers. A different perspective on the role of standards that is difficult to ignore was put forward by Fraser (1997), who argued that If standards are raised without an accompanying commitment for funding then these standards are little more than a way to continually disenfranchise a portion of society, ensuring the necessary human capital for low level jobs and a subsistence living. There can be little doubt that if there was a significant increase in the number of high school graduates it could result in social dynamite, the building of profound social discontent. (Fraser, 1997, p. 26) In this view, a large number of people living at subsistence levels is a stabilizing influence in the social order of most liberal democracies so long as the disenfranchised sectors of society do not gain access to a political voice. Many policymakers clearly abhor this view, as is evident by the focus of this dissertation being a ministry-sponsored process of devising an ABE accountability framework that could incorporate non-academic indicators of student and program success. Had the framework been implemented despite the change in government, this would have encouraged a more eclectic demonstration of the ways in which ABE programs and learners are successful. A modestly speculated outcome would have been greater access that enabled more, not fewer, learners to succeed in their educational pursuits. Designing Accountability Frameworks A politically realistic accountability framework for adult basic education cannot dismiss measurable outcomes, given that they are the mainstay of policymakers who are likely not educators, but in the interests of inclusion and quality the process cannot be limited to them 73 either. Rather, it would have to provide the ministry with an assessment process to ensure that ABE providers are effectively utilizing government funding for the development and implementation of successful ABE programs for adult learners, while also providing the field with an advocacy mechanism capable of legitimizing the success of their programs. Striking this balance requires that the framework's design be guided by the canons of accountability conventions as outlined above while recognizing the unique nature of ABE programs and the learners attracted to them. This section begins by discussing the basic constituents and design requirements of an accountability framework that is capable of respecting and incorporating the different needs and value systems of the stakeholders involved in the provision of school district ABE programs, then turns to briefly outline some useful strategies to guide the development and implementation of an accountability mechanism. In a previous accountability era, Glass (1972) cautioned that educational accountability tended to promise more than it could deliver: This was because too often educational accountability was defined through rational empirical "retributive" approaches which did not adequately address the contextual complexity of education and frequently resulted in impotent remediation strategies that did little to nurture meaningful and long-term school improvement (p. 636). In an effort to arrive at an accountability model that might genuinely improve schools he analyzed six models of educational accountability that included input-output analysis, accreditation, program planning and budget systems, vouchers, and performance contracting (p. 637). The models utilized descriptive data that correlated pupil performance with school programs, personnel, plans, objectives, and operations. While he found these traditional accountability models weak in a number of areas, what was most troubling was the subtext of "retributive accountability" that underscored the endeavours and led to attempts to "credit or punish individual teachers, 74 administrators, or school districts on the basis of non experimental, correlational evidence concerning their contributions to the welfare of students" (p. 637). A more successful approach to accountability, Glass (1972) argued, would have contributed to improved student performance through institutional innovations. Such a model would be mindful of the social nature of education and the complexity of particular school settings. School improvement would remain considerate of comparative evaluation but shift its focus to (1) continuous and valid experimentation, (2) understanding the diverse requirements of different regions, and (3) evaluating alternative interventions within the school setting to improve student performance and wellbeing. These changes in addition to implementing random assignments and single blind testing would, in his view, assure that "schools are following the only feasible course toward greater accountability for improving pupil performance and well-being" (p. 639). Glass's (1972) critique of educational accountability demonstrates the problematic nature the term faced over thirty years ago. Arguably, little has changed since. His call for an approach to educational accountability that considered the complex social context of education and schools has resurfaced in more recent research. These arguments continue to advocate accountability frameworks that are not limited to empirical retributive approaches to school improvement. Macpherson (1996) put forward a "systems" (or system-wide) approach to educational accountability which would be inclusive of both traditional and nontraditional standards for measuring performance. He conceived educational systems to be multiple and complex "networks that comprise socially constructed beliefs and feelings, a moral economy of norms and values, and empirical knowledge" (p. 103). His understanding of the process for determining 75 accountability in this network of eclectic institutions is at some levels fairly conventional, as it involves establishing clear purposes, defining performance indicators, gathering performance data, and giving that data prominence in planning (Macpherson, 1996, p. 91). The model furthermore takes into consideration more traditional indicators such as achievement tests, and is designed to collect and report objective data about role-related behaviours that are evaluated against economic, social, and other criteria used to improve the system (p. 82). Macpherson's model begins being a bit unorthodox by giving voice to diverse definitions of accountability held by the various stakeholders of public education whose perspectives are honoured and built into the evaluation criteria and processes. This "interperspectival" position takes into consideration the many different knowledges, values, and opinions of those with a stake in the process. It is a proactive approach to accountability - politically expedient but also genuine in its belief that plural alternatives result in criteria and processes that reflect a more knowledgeable, holistic, and effective approach to school improvement. Macpherson's model gives freedom of expression to the dual nature he recognized to be embedded in the concept of accountability which affords it agency "to both reproduce and transform societies' values" (p. 80). Accountability processes and criteria are thus "a major means by which the moral economy of an organization is established, reproduced, and, if educative, transformed" (p. 92). Data gathered through accountability criteria and processes result in action that either forces people to accept embedded values or enables them to review, revise, develop, and strengthen the values they bring to the assessment enterprise. In this sense, accountability policies have the potential to create "psychic prisons" or "emancipatory conditions" (p. 92). 76 Macpherson's analysis of the methodology and epistemology of educational accountability was made in an effort to derive informed policies that "end not with a rational and functional theory of appropriate behaviours based on objective facts, but with a practical set of policy options that best accommodate empirical, subjective, and normative data" (p. 81). Emphasizing the importance that stakeholders understand the moral consequences of determining criteria and designing processes for measuring accountability, he aptly observed that accepting responsibility for education in inclusive, collaborative forums not only "coheres with the need for a consequentialist concept of accountability criteria and processes" (p. 103) but is fundamental to establishing "legitimacy in policies and practices in a complex democracy" (p. 80). These aspects are also, he suggested, essential for being accepted by educational practitioners, and thus for being successfully implemented. Merrifield (1998) took the notion of collaboratively designing accountability frameworks further into the field. She advocated a "performance accountability" model based on "mutuality" - a cooperative approach to accountability that engages all players in clarifying expected outcomes, ' defining indicators of success, building capacity, developing a common vision, determining goals and customer expectations, and designing effective means of monitoring, producing, and improving performance (p. 1). Predicated on values of inclusivity, Merrifield's model places particular importance on designing efficient and effective information flows between stakeholders to raise their awareness, enable them to develop and apply the most appropriate measurement tools possible, and facilitate their capacity to hold each other accountable for how the ultimate findings will be used (p. 10). 77 Four principles guide Merrifield's performance accountability model which are based on a "bottom up" rather than "top down" philosophy. These include "agreeing on performance, building mutual accountability relationships, developing the capacity to both perform and be accountable, and creating] new tools for performance" (p. 8). Good performance is not a technical question but is "inevitably in the realm of values" (p. 9). It includes the full diversity of a program's purposes, with indicators developed through public debate engaging multiple stakeholders. Developing performance indicators in a mutual accountability relationship entails greater transparency, increased flows of information, and room at the table for teachers, learners, and a full range of other stakeholder groups who may not have participated in developing performance indicators in the past (p. 9). Particularly if learner participation is included, there may be an opportunity to address inequalities in power and access to information, and strengthen the voice of adult learners in the system. This "learning organization" is built on the capacity to perform by focusing resources on quality rather than quantity, staff development and training, technical support, and the use of performance data for continuous improvement. Accountability systems work best if stakeholders can agree on what success looks like (Merrifield, 1998). For adult basic educators, this strikes at the heart of the matter. What is our concept of ABE? The concept has shifted over time. The autonomous model conceives of ABE as a discrete and fixed set of skills, transferable from one context to another. The ideological model sees ABE as practices that are sensitive to social context and inherently associated with issues of power and access. With such diverse understandings, notions of success are multiple. The diversity of opinion grows exponentially when "external" constituencies from government and the community enter into the design process (Ben Jaafar and Anderson, in press). A single definition of success - gaining the GED (General Educational Development certificate), for 78 example, or getting a job - excludes learners who have different purposes (Street, 1984). Merrifield's model overcomes this difficulty by developing several types of indicators to track performance over time. The model is based on the assumption that learning is at the heart of ABE - an activity standardized tests cannot on their own adequately demonstrate. In Merrifield's framework, input indicators provide information about the capacity of the system and its programs, while process indicators follow organizational and educational processes that support performance. Output indicators measure the more immediate program results such as course completions, as well as long-term impacts (Merrifield, 1998, p. 54). This accountability framework uses external standards or criteria that involve performance measures of individual student learning in real-life contexts. Central to ABE accountability in this concept is the ability to track student learning, demonstrate what has been learned, compare learning across programs, and judge learning against external standards (p. 7). Research, evaluation, and monitoring technologies are used to meet the needs of different stakeholders. These approaches to gathering, analyzing, and using information are based on different kinds of data that serve different purposes. Using them in appropriate ways would assist ABE providers in developing a defensible accountability process (p. 11). Merrifield's approach resurfaced in Thompson (2001) who also raised the possibility of developing authentic, standards-based reforms that were concerned with equity and capable of capturing the latent, more subtle outcomes expressed among ABE learners. Through this approach, common learning standards are collaboratively determined by teachers and other stakeholders (including parents in conventional K-12 systems, students in ABE systems, community members, etc.). Students have numerous ways to demonstrate how they have met the standards that determine their educational assessment. Under this standards-based approach the system invests heavily in professional development for teachers and administrators to support their efforts in reaching the standards. The intent of the approach is to promote a learning environment "dedicated to helping all students reach their intellectual, social, and personal potential" (p. 360). This, Thompson (2001) argued, is in contrast to the educational purpose of high-stakes testing which is to raise test scores. The accountability frameworks outlined above are inclusive, socially progressive approaches capable of incorporating the more traditional accountability conventions introduced by Boyle (1981) and Boone (1985). A hybridized model that accommodates some of the widely varied perspectives of the different interested parties involved may face a better chance of acceptance and unimpeded implementation. If diverse stakeholder perspectives are integrated successfully the viability of an accountability framework's design and adoption may be strengthened. An invaluable strategy in this sense may be to adopt suggestions for implementing accountability mechanisms put forward by the main funding agencies concerned with A B E programs. In British Columbia, during the timeframe of this study, the Office of the Auditor General had been particularly productive at enumerating the means to measure accountability. Incorporating some of these or other government departmental ideas into a school district model might enhance its philosophical alignment with accountability conventions and increase its likelihood of acceptance by the Ministry of Education. The strategies suggested by the auditor general's office cover program objectives, program activities to determine how the objectives are to be achieved, and program performance measures to demonstrate the extent of achievement (British Columbia, Office of the Auditor General, 1996). Performance assessments include input measures which are often related to resources used by the activities; in government these are usually expressed as money spent or number of employees involved. Quantitative and qualitative output measures of program activities are also included. Outputs are distinguished from outcomes, as the former are closely related to program activities while outcomes reflect societal impacts (ibid.). Performance measures follow the goals and objectives of the organization, are limited to a reasonable number, and are simple, meaningful, and manageable. Evaluation may be summative, assessing the final product, or formative, improving the product on an ongoing basis. Evaluation tasks include determining what to evaluate, how to collect the evidence, and how to develop and judge the evaluation criteria (ibid.). The recommendations provided by the Office of the Auditor General represent a "hegemonic discourse" in that they are top-down government directives that do not include a plurality of views that could take into account "the uncertainties and risks associated with educational practice" (Grundy, 1992, p. 157). Grundy's (1992) work on counter-hegemonic discourses offers a means to balance the perspectives of the auditor general's office. Grundy's view is based on understanding education as praxis - a social practice of moral action with conceptions of what is moral being collaboratively determined by the stakeholders involved. Whereas traditional, technically oriented accountability mechanisms are concerned with "bringing about" pre-specified ends by acting on objectified beings, praxis is a deliberative form of social action taken with other actors that have a subjectivity of their own. An accountability framework guided by praxis implies democratic rather than top-down management, cooperating with rather than attempting to control the environment, and incorporating pluralistic understandings of performance that work towards the continuous improvement of education. It reflects the interdependent realms of learning, teaching, and leadership, and seeks to understand education as a social action that takes place through interaction. Community, teacher, and learner 81 empowerment become key elements fostering an environment where continuous change - the ever-present constituent of accountability - can take place (p. 168). Accountability structures defined through praxis imply that judgements about the meaning or worth of human action need to reflexively consider the particularities, persons, situations, and events that formed the action. The idea of plurality in the discourse of praxis suggests regional, demographic, ethnic, and cultural differences would be included in decision-making processes involving shared responsibility for action. These complex context-specific factors are often overlooked in conventional accountability mechanisms which tend to encourage uniformity in assessment practices and standards of measurement. As Evans et al. (2000) observed, an effective accountability policy would "change the focus of evaluation criteria from one based on averages to one relying in addition on the distribution of access and achievement" (p. 19). An accountability format that incorporated more nuanced understandings of the different kinds of learners being served might better capture the individual characteristics and experiences that influence school success. A more descriptive and personalized type of assessment would include background variables such as age, employment, years of schooling, country of origin, languages spoken, changes in life circumstances (such as job, citizenship, and self-sufficiency) as well as academic assessments through indicators such as current literacy/numeracy level and shifts in learner goals. This data might be valuable in better understanding the profile of learners who did not experience success within the structures of the traditional K-12 system. The diverse sociopolitical values and understandings of the purpose and process of adult basic education that different constituent groups bring to the table tends to create a power dynamic that arises whenever there is a heterogeneity of preferences, tastes, endowments, and expectations 82 (Drazen, 2000). Coalitions and compromises may then be created to breach conflicts of interest and enable some individuals or groups to achieve outcomes that more or less reflect some of their objectives (ibid.). Ben Jaafar and Anderson (in press) provide insights into understanding the power plays at work between different factions in the development of an accountability framework. Elaborating upon Spencer's (2004) politics of accountability paradigm which drew on Blackmore's (1998) model, the authors suggest Canada is characterized by two alternative, occasionally competing philosophical approaches to educational accountability which "co-exist in relation to different dimensions of the education system, and sometimes intersect in unpredictable ways on the same components of the system" (Ben Jaafar and Anderson, in press, p. 6). These two approaches to accountability are expressed as the "economic-bureaucratic" and "ethical-professional" paradigms. The primary concern of the economicrbureaucratic accountability orientation is with outcome not process. Emphasis is on utilitarian goals of maximizing benefits with limited resources. This is a results oriented form of accountability, wherein process is validated by performance outcomes (Ben Jaafar and Anderson, in press, p. 7-8). The ethical-professional accountability paradigm, by contrast, is process-based, that is, "means are emphasized over the ends of schooling" (ibid., p. 9). Responsibility is not ascribed to individuals, but is considered a moral obligation of all professional stakeholders. This perspective emphasizes process not outcome, and values activities that enable stakeholders to engage in shared processes to define and contribute to common purposes established with reference to collectively determined professional and ethical principles (ibid., p. 9). 83 This dichotomy of paradigms, Ben Jaafar and Anderson contend, should not be conceived as mutually exclusive diametrically opposed extremes but "rather as co-terminus ideological orientations to the same educational accountability field that co-exist in a more fluid and dynamic state of differentiation and interaction" (ibid., p. 9). While the economic-bureaucratic paradigm tends to be associated with hierarchical and market-centred models (or in this case with politicians and government bureaucrats) and the ethical-professional approach is preferred by education professionals, the authors argue that no group adheres to one philosophical approach to the exclusion of another. Rather there is an overlap in practice. Ben Jaafar and Anderson suggest that the economic-bureaucratic accountability model predominates in Canada with some acknowledgement granted to the ethical-professional approach for reasons of political expediency. Nevertheless, the authors also indicate the reason the emergence of "robust and powerful [ethical-professional accountability] practices have yet to be realized" is possibly due to these practices not yet having been conceived and voiced (p. 34). This result might be due to the absence of practitioners being genuinely engaged in the design of an accountability framework. Summary The ambiguous mandate of school district ABE programs creates unique challenges for the design and development of an ABE accountability framework for school district providers in British Columbia. Adult learners and the programs where they experience success vary greatly throughout the province, making it desirable to design an equitable accountability framework capable of assisting districts to explain the value of success in their programs while recognizing that each program is unique. The political, economic, social, and technical dimensions of 84 educational accountability support a conventional approach to building an accountability mechanism which includes measures for determining performance and assessment that have become increasingly controversial (Glass, 1972; Macpherson, 1996; Delandshere, 2001). Accountability designs may well be undermined by the influence of unnoticeable structures and power dynamics that manipulate and bias stakeholder conceptions of the purpose of education and the attributes exhibited by students and programs that are worth measuring (Scheurich, 1994; Macpherson, 1996). This is especially the case in the design of an ABE accountability framework which operates within the outcomes and standards dictated by a K-12 mandate. Although research on ABE students highlights their multiple goals and diverse academic profiles, the accountability structures that apply to ABE programs rely on standardized testing and course completion rates which neither allow for individual, geographic, and demographic differences nor address issues of self-esteem, motivation, and wellbeing. Academic success is the only factor deemed worth measuring. Unfortunately, the narrow focus on academic success leads to the increased social exclusion of populations most at risk. Policies supporting evaluation criteria that recognize the value of learning outcomes extending beyond academic and systems-driven measures of success would counter the trend toward marginalization. Rather than replicate the traditional structures that initially marginalized many ABE learners, progressive accountability models that support the needs of disadvantaged learners and the health of society at large would consider that the development of core skills was contingent on the capacity to develop self-esteem and empower voice. Macpherson's (1996) and Merrifield's (1998) models appear particularly capable of addressing many issues surfaced in the literature which argued for more comprehensive and authentic forms 85 of accountability that better represented individual learner success within educational programs (Bingman and Elbert, 2000; Cibulka, 1990; Delandshere, 2001; Eisner, 2001; Glass, 1972; Grundy, 1992; House. 1972; Knox, 2002; Thompson, 2001; Torrance, 2001; Wrigley, 1998). Both approaches could incorporate accountability conventions surfaced in Boone (1985), Boyle (1981), and the British Columbia Auditor General's report (1996). At the same time, both could potentially address the issues of praxis, inclusivity, and mutuality which surfaced in the literature debates concerning educational accountability (Bingman and Elbert, 2000; Cibulka, 1990; Delandshere, 2001; Eisner, 2001; Glass, 1972; Grundy, 1992; House, 1972; Thompson, 2001; Torrance, 2001; Wrigley, 1998). Merrifield's (1998) model, however, does not address a world where conflicting interests, contested visions, and competing values are often the defining features of heterogeneity and collective life (Drazen, 2000, pp. 1-2). The mutual and inclusive decisions taken in building an A B E accountability framework will therefore be critically analyzed and interpreted through a sociopolitical lens that is aware of compromise, coalition building, and the construction of constellations of networks as possibly the normative practices of educational policymaking (ibid.; Ben Jaafar and Anderson, in press; Macpherson, 1996). 86 Chapter Four The Evolution of an Accountability Framework This chapter examines how an accountability framework for school district adult basic education programs in British Columbia evolved over the course of three years, from late 1998 to the Fall of 2001. The account chronicles the framework's growth and development from a chaotic mass of conflicted demands, partial perspectives, and fragmentary ideals into a coherent and flexible instrument shaped by continuous information flows and feedback mechanisms, including research, field participation, and the expertise of members from the Ministry of Education's ABE accountability committee. The design process cumulated in an accountability framework that reflected an inclusive, socially diverse understanding of ABE that interwove socio-educational concerns surfaced in the literature with conventional accountability structures related to more pragmatic understandings of educational accountability, particularly for adult basic education. The chapter describes the design process, beginning with an outline of the information and circumstances impacting the prospect of building an accountability framework for school district ABE programs when it became clear that designing a mechanism to be held in common with the college sector was unfeasible. We then review the intelligence-gathering work that contributed to a more informed design and conclude by discussing the framework's intended impact. A Conflicted System Multiple Urgencies Throughout their history, British Columbia's school districts have not been a model of consistency with regard to ABE accountability practices, as noted in chapter two's description of 87 Educom's province-wide survey of school districts. Although the survey's methods were imperfect and the results confusing, the course completion rates reported by Educom raised cause for concern. The survey's immediate impact was that in the Spring of 1998 each school district's adult student enrolment was capped at its FTE for the 1997/98 school year, and the provincial cap was frozen at 12,776 FTE. The enrolment cap became an issue of significant concern among school districts as many of their adult education programs were growing, and some districts were enrolling more adult students than they received funding for. For the 1999/2000 school year, school districts reported 12,498 adult FTE, but the ministry only funded 11,722 FTE. Provincially, the school districts had exceeded their caps by 776 FTE, and were not receiving funding for the increased enrolment (My analysis of MoE 1701 enrolment data). The British Columbia School District Continuing Education Directors' Association (CEDA) met with the Ministry of Education in December 1998 to discuss a process for removing the adult enrolment cap and the possibility of reallocating underutilized adult FTE funds to districts that exceeded their cap. A number of issues concerning school district delivery of ABE became evident from this discussion. It surfaced, for example, that although all districts received the same amount of funding for adult students, these funds went into general revenue and allocations to adult programs varied from district to district. Some allocated more of their adult revenue to the development of adult programs while others allocated less. Through conversation and an air of collaboration encouraged by CEDA's powerbrokers among other school district representatives, a consensus arose that districts demonstrating irregularities should be audited. There was less consensus, however, on what these irregularities might look like, what norms they should be based on, and how performance in ABE programs and subsequent success indicators might be determined (My fieldnotes, MoE/CEDA meeting, December 21, 1998). 88 The ministry and CEDA agreed that district enrolment and relevant success measures should form the basis for additional funding. Performance would be a key indicator to the deployment of underutilized funding, but there was little agreement on what constituted successful performance. The MoE indicated that if the cap was to be removed higher accountability would be required based on standardized reporting of FTEs. Districts could also apply for funding if their populations of ABE learners grew, and to demonstrate accountability they could undergo an audit. An accreditation process was put forward as a possible outcome of the auditing and accountability process, which, it was agreed, might also influence districts to allocate more ABE funds to ensure programs would survive the scrutiny of accreditation. In this initial year and throughout my secondment I found that by being both part of the ministry and CEDA, I was able to offer insights into how CEDA might move in ways that were more supportive. CEDA's tacit knowledge, acknowledgement of district accountability problems, and willingness to cooperate with ministerial initiatives increased its influence in the MoE during the NDP years, a powerbase it now tenuously holds with the current Liberal administration due to an erosion of support for ABE, as will be discussed in Chapter Six. At the time of this study, CEDA's collaborative disposition enabled it to press for an accountability framework that would increase the transparency of school districts without undermining the progress districts had made in ABE programming. The most striking feature that rose from the discussions between CEDA and the Ministry of Education over the removal of the cap was the lack of structure regarding school district reporting of adult FTE and the overall absence of accountability structures for school district ABE program delivery. These initial discussions laid the basis for a joint resolve to work 89 towards a more thorough understanding of school district delivery of ABE programs and a means for measuring accountability within these programs. Partial Perspectives The discussions between CEDA and the Ministry of Education exposed considerable ambiguities concerning school district delivery of ABE programs and revealed that there was little ministerial direction or understanding of ABE program delivery. As these talks occurred when the MoE was attempting to build a common model for measuring the accountability of ABE programs in collaboration with MAETT, the school districts, and the colleges, the MoE sought to better understand this area of its portfolio. To improve the ministry's understanding of adult learners involved in the K-12 system and to inform future ABE accountability initiatives a K-12 Adult Learner Outcome Survey was launched in April and May 1999. This survey, however, was remarkably limited. As the survey cohort was confined to students who would complete grade 12 through either the regular or adult graduation programs in September 1997 and who did not return to the system in August 1998, the survey provided useful information on only a fraction of ABE students.1 Because the survey's research design had been narrowly focused on understanding the outcomes of ABE graduates who went on to further education, it did not provide generalizable data that could help understand the majority of K-12 learners or better inform ABE accountability initiatives. The cohort chosen, representing only students who had registered with the MoE to graduate in September 1997, was especially problematic because in 1997 students working on the adult graduation program did not have to write provincial exams and did not have to register with the MoE. School districts were able to issue the Adult Dogwood at the district level and many did not report their adult graduates to the ministry. There were no 90 available data to determine how many students would have fit the student profile but were never registered with the MoE. Additionally, the survey did not capture ABE students whose motivation to enroll in ABE was academic upgrading and not graduation. The survey did not accurately portray a provincial profile of the majority of ABE students, and especially not of those who were the hardest to serve. Its limitations were readily apparent when the survey cohort profile was applied to the ABE programs that fell under my aegis prior to my secondment. Those programs enrolled approximately 1000 full- and part-time adult learners, of whom only 18 would have been included in the 5,343 student cohort used in the survey. The average age was over thirty years old, the vast majority were unemployed and on some sort of social assistance, approximately 25% were First Nations learners, over 40% were working on coursework that was at a literacy level, and a high percentage had learning difficulties. In some years attrition rates approached 70%; however, the total enrolment figure of 1,000 students remained constant, and there was often a wait list. The ABE student profile in my school district was typical of many large rural districts in British Columbia. The majority did not reflect the student profile of the Outcome Survey cohort. As was also evident in the literature review, students who enroll in ABE programs are characterized by highly diverse backgrounds (e.g., Beder and Valentine, 1990; Quigley, 1990; Mezirow, Darkenwald, and Knox, 1975). For the purposes of the framework's design, the survey offered a very narrowly defined picture of ABE learners in British Columbia's school districts, and was indicative of the cost-savings measures the MoE often took toward understanding adult basic education. Aside from the theoretical studies that I brought to the design the survey was at the time the only research the ministry had to contribute toward drafting a provincial accountability initiative in common with MAETT and the college sector. 91 Fragmentary Ideals Consisting of MoE and MAETT representatives, an ABE Transitions Project was formed in the Spring of 1999 to develop a standard quality and accountability framework for both school district and college providers of adult basic education, as described in Chapter Two. Although a common framework could not get off the ground because of systemic differences, the Transitions Project initiated valuable discussion about the goals and objectives of ABE. I was present at these talks, as the seconded school district senior administrator charged with contributing theoretical research on educational accountability and providing input into a draft of a common framework. These discussions formed an enlightened part of the prehistory of the school district's ABE framework, as they enriched its design. A summary of the goals and objectives that surfaced during the MoE/MAETT subcommittee's discussions aimed at creating a common framework is provided in Table 4.1. TABLE 4.1 GOALS AND OBJECTIVES OF ADULT BASIC EDUCATION GOAL 1: FURTHER EDUCATION GOAL 2: EMPLOYABILITY SKILLS GOAL 3: LIFE MANAGEMENT SKILLS OBJECTIVES: pre-requisite skills pre-requisite courses pre-requisite credentials INDICATORS: enrolment performance OBJECTIVES: OBJECTIVES: provide academic skills assist in discovering present provide personal management skills resources provide teamwork skills utilize present resources more effectively " provide ways to develop/use new resources INDICATORS: communicate think learn for life positive attitudes and behaviours responsibility adaptability ability to work with others INDICATORS communication problem solving healthy living Source: British Columbia, MAETT/MoE, 1999. 92 The Transitions Project defined ABE as a program that provided adults with access to preparatory courses and skills training which might range from basic literacy to secondary school completions and lead to further education, employability, and/or life management skills. It put learners at the core of the education process, and noted that the goals of the framework needed to relate to the goals of the learner. The subcommittee emphasized that evaluation be understood as a process that focused on the strategies that programs had in place to achieve the aforementioned goals. Accountability was defined as outputs and outcomes. Success was to be measured in terms of student progress, performance, and attendance. The subcommittee noted that the adult student profile was an important consideration in determining what constituted success in ABE programs. Success was viewed as a contestable term among committee members, leading ABE learners to be redefined from "students who did not achieve success in the K-12 system" to "students for whom the K-12 system was not successful", signifying a pivotal change in perspective that placed increased responsibility for student performance with K-12 structures (British Columbia, MAETT/MoE, 1999). The Transitions Project revitalized the ABE Program Quality Framework (Appendix 1) which was based on recommendations from the 1994 Ministry of Education and Ministry of Skills, Training and Labour Joint Committee on ABE. This framework was an attempt to address MoE's and MAETT's desires for an accountability framework common to colleges and school districts (British Columbia, Office of the Auditor General, 1994). The framework was piloted in the Spring and Fall of 1999 by a facilitator in two school district and college regions utilizing a consensus-based process that attempted to bring together a variety of ABE stakeholders in the college and school district sectors (admissions/advising staff, instructional staff, volunteers, administrators, learners, etc.). Although the piloting process promoted valuable discussion around ABE program quality, it did not address the complex issues concerning indicators of student success, nor did it address those aspects of the framework that were dependent on funding. As discussed in Chapter Two, the proposal for a common framework posed profound and inevitably insurmountable challenges embedded in systemic funding differences. This led the MoE to hold the seconded school district director (myself) over for another two years to initiate a single-ministry committee-led design of an A B E framework. The first step in the process was a province-wide study of school district operations, known as the Provincial School District Review, which I was responsible for, and which provided information that would ultimately lay the basis for the development of an accountability framework for adult basic education programs offered through school districts. The Provincial School District Review In January 2000 the Ministry of Education brought together members from the ministry and the field to form the MoE adult basic education accountability committee. Comprised of two ministry officials with A B E expertise, a senior school district director who represented CEDA, and my involvement as a secondee, the accountability committee was mandated to develop an accountability framework for school district ABE providers. The committee's first move was to attempt to acquire a more comprehensive understanding of school district adult basic education programs by initiating a provincial review of ABE operations falling under the aegis of the MoE The committee met bimonthly to provide feedback on the evolution of the A B E accountability framework and to make recommendations for future directions. 94 As the seconded school district director of adult education it was my responsibility to go into the field, examine school district ABE program operations, and report my findings back to the ABE accountability committee. During the Spring of 2000 and 2001, I reviewed seven large districts in central and northern BC and two in the Lower Mainland that were involved in the delivery of school district ABE programs. As it was not possible to review all school districts, those selected were the largest providers of ABE in the regions chosen for the review under the assumption that large districts would comprise the most diverse student populations. The review's primary intent was to gather information on the diversity of ABE program delivery to inform and guide the ABE accountability committee in the development of an accountability framework. A secondary intent was to signal to the field that the MoE was serious about an accountability process for school district providers of ABE. The provincial review revealed a number of diverse and complex issues involved in school district ABE delivery. Issues that were particularly important to a district or frequently voiced among multiple districts were summarized in an overview to the MoE adult basic education accountability committee. These issues highlighted some commonalities but many singular problems unique to each school district's ABE programs with regard to enrolment, admissions, attendance, progress, student records, teacher qualifications, fiscal accountability, hours of operation, and modes of program delivery. These issues became foundational to the development of an ABE accountability framework capable of reflecting the diverse nature of school district ABE delivery. (See the section "Informing Elements" below in this chapter for a more in-depth look at some of these issues.) 95 The provincial school district review not only uncovered problems that the districts faced but disclosed anomalies in the districts' delivery of ABE. These were impacted by, but not limited to, factors such as student ability, socioeconomic status, regional demographics, hours of operation, teacher certification/qualification, physical facilities, course selection, and curriculum delivery format (i.e., teacher-driven, computer-driven, or both). There appeared to be considerable variation in the defining, accounting, and reporting practices that each district adopted. These discrepancies were sufficient on their own to underscore the need for the development of a province-wide accountability framework. The imperative for an accountability process gathered steam, however, as the onset of the provincial review coincided with an abrupt dip in enrolments. Provincial school district enrolments for the May 2000 Form 1701 adult student funding count dropped 1,662 FTEs below the adult provincial cap of 12,776 FTEs. This was the second year in a row that FTEs had fallen below the cap, and it opened opportunities within both the MoE and the school districts to influence discussions about A B E accountability and the subsequent evolution of the framework. The previous year, in May 1999, school district A B E enrolments had fallen by approximately 278 FTEs below the provincial cap. When the 2000 enrolment figures were released, CEDA was in the midst of working with the ministry to develop a process for the reallocation of underutilized adult FTE funds and the removal of the provincial adult FTE cap. When the MoE initiated the provincial A B E review, coincidentally several districts already concerned about the enrolment cap for adult learners were now below their cap. However, there were still a few districts reporting an increase in adult FTEs beyond their cap. The significant overall drop in adult enrolment created a revenue surplus of approximately $6.5 million and provided an opportunity for districts that had exceeded their enrolment to apply for additional funding. The 96 ministry wished to share its largesse because if it did not find a way to spend the surplus, the Treasury Board would claw it back. In May 2000 the ministry sent a letter to all school districts indicating that as the ministry continued to review its accountability process to better ensure equity of access for all learners, it would consider reallocating underutilized FTE funding to districts that had exceeded their cap. Eligibility would be determined through a review of the districts' adult programs conducted by an external review team comprised of Ministry of Education staff (from School Finance and Field Services departments) as well as external school district directors of adult education in current practice (including myself). A product of the combined events of the provincial review and the drop in adult FTEs, the external review process was mobilized by several concerns. At one front, it was intended to address the field's concerns about the preclusion of adults from ABE programs because of district caps on adult enrolment, and thus it became a short-term accountability process that provided districts an opportunity to apply for additional funds. At another front, the external review provided the MoE another opportunity to gather information on school district delivery of ABE, though only on districts that had applied for additional FTE funding. It thus served to assist the Accountability Committee in making final recommendations for a long-term accountability strategy and framework design that could lead to the cap's eventual removal. In response to the prospect of additional funding, six districts invited the Ministry of Education external review team to conduct reviews of their adult education programs. In September 2000 the districts that applied for additional funding were sent a list of expectations regarding the 97 review process (Appendix 2), and the external review team was provided with a modified check list that included additional questions (Appendix 3). After the review team met with district officials to go over the process, the external review began in the Fall of 2000, simultaneous with the ongoing provincial review. Six districts were visited. With a school district designate present to facilitate the process, five specific criteria were examined: attendance records; progress reports or student records; consistency in FTE monitoring; learning plans demonstrating how the coursework led to high school graduation; and allocation of adult FTE dollars (Accountability Meeting Minutes: February 8, 2001, BCIT Downtown; February 22, 2001, SFU Harbourside). These criteria were largely derived from issues surfaced in the provincial review that the accountability committee determined were satisfactory initial indicators of program quality. Each school district provided the external review team with space to work and hard-copy evidence that showed enrolment figures; pupil-teacher ratios; specific ABE costs (personnel, technology, leasing, etc.); teacher timetables; hours of operation; examples of forms used (e.g., Individual Education Programs, tracking forms, attendance forms); and delivery methods. The review team completed each district review in a single day. Following the review, the team debriefed, discussed their results, and made recommendations for approval or denial of the district's application for funding. The recommendations of the external team were forwarded to the Assistant Deputy Minister for final decision in December 2000.2 The results of the external review confirmed many findings of the longer, simultaneously running provincial review. These results became the basic scaffolding for building the ABE accountability framework. 98 Constructing the Framework The accountability framework was constructed over the course of the year and a half that the provincial review was underway through the bimonthly meetings of the ABE accountability committee. This policymaking process was an interweaving of theories and experiences which were collected concurrently, cumulatively, and in collaboration with the accountability committee and many others in an attempt to evolve an educational paradigm that better met the needs of the learner, the field, the bureaucrats, and the politicians. By combining contemporary and conventional approaches to accountability and policymaking 1 hoped to develop an argument that had "the capacity to persuade, cajole, threaten, and/or endear" (Gardner, 1990, p. 42) the ministry with regard to the purpose of ABE, the needs of its students, and the necessity of adequately funded ABE programs. As the provincial review was implicated in the framework's construction, the same committee oversaw both mandates. Comprised of two ministry officials with ABE expertise, a senior school district director representing CEDA, and my involvement as a secondee, the accountability committee's bimonthly meetings provided a venue for discussing the findings of the provincial and external reviews and the ideas surfaced from other research. The meetings often had in attendance outside experts invited by the Ministry of Education who provided informed input into the framework's logistics and design. The "political immediacy" of this policymaking process evolved as a result of what 1 call "resident expertise", encompassing the committee's combined professional expertise enriched by contacts from the field, the ministry, and academe. Philosophically, the committee members were concerned with the design and implementation of an ABE accountability framework that would reduce existing oppressive structures and afford 99 adult learners a less restrictive and more learner-centred pathway to graduation. Policy grounded in this philosophy aligns itself with Robinson's (1994) discussion on critical research being dedicated to understanding and altering the conditions that prevent people from living fulfilling lives, and Marshall's (1997) contention that critical theory is an analysis of power, policies, and structures that restrict access to knowledge and concentrate power into the hands of the elites. The conceptual perspectives that informed the committee's policymaking process were based on these tenets of critical theory. As will become evident further in this chapter when we examine the section, Form, Function, and Purpose of the ABE accountability framework, the mechanism's architects were particularly concerned with building an "emancipatory" approach to student assessment. This meant that we sought to understand student success in ways that respected but were not restricted to the confines of a results-based, economically-driven accountability model. We instead looked to a process that had the potential to encourage democratic discourse among stakeholders to arrive at a mutually agreed upon ethical-moral understanding of educational accountability. We believed that an emancipatory understanding of student assessment would support the learners in their process of self-understanding, intervention, and transformation. We also thought that to initiate this process required an approach to educational accountability that would enable ABE practitioners to widen their understanding of existing notions of educational accountability, possibly resulting in new conceptual formations that recognized and valued non-academic understandings of student success. Based on the research, the ABE accountability framework's authors hoped to achieve a more holistic understanding and recognition of student success that could enable the students and practitioners to emancipate themselves from accountability structures that were too often guided by the influence of social regularities such as governmentality, professionalization, and 100 reification (Scheurich, 1994). In this sense the ABE accountability framework would serve as an accountability model that embraced the transformative potential of educational accountability practices (Macpherson, 1996). The policy design processes that the committee was involved with were "cutting edge" in that relevant research tended not to be available to guide us on a practical level. The design of the framework had no precedent as far as its architects were aware. As committee members were unable to find related research on democratic, emancipatory, or progressive accountability structures, the ABE accountability framework's design and implementation depended on the expertise of the field and ministry practitioners making up the committee. Their practical know-how and related academic strengths provided some relevant formal (or codified) understanding related to the problem, but by and large we worked under the tacit, intuitive, and instinctual circumstances of acquiring knowledge. As I worked with the ABE Accountability Committee to design the framework I was continually building upon theory and experience as a secondee to ensure the framework would be marketable to political and practitioner interests. The design was a constant struggle - not an internal struggle but a practical one focused on how I could achieve a nontraditional yet acceptable framework of understanding for ABE accountability. The more I leant toward the non-academic indicators, the more apprehensive I became that the plan would lose and become an impossible policy to create, let alone implement. The findings I brought from the provincial review were repeatedly vetted through the ABE accountability committee's resident expertise. This resulted in a framework design that was 101 consensually based with respect to what elements would be included and how the mechanism would function. As the provincial review's results made it clear there were few operating standards in the school district ABE sector, the committee proposed a number of technical criteria that could be implemented to better ensure accountability in the provision of ABE. These included quantifiable data on attendance, student progress, teacher qualification, pupil-teacher ratio, technological characteristics, physical facilities, and district allocation of ministry provided FTE funds for adult education. However, the school districts involved in the provincial review also made it known that the standards for one district might not be transferable to another because of the highly variable demographic, cultural, and socioeconomic conditions found across British Columbia's regions. We now turn to address some specific instances that comprise the basic elements of the framework's form and illustrate the controversial, distinctive, and often regional nature of the issues raised by the provincial review. Informing Elements The provincial review surfaced a number of complex issues that made it clear a highly sensitive and open-ended accountability instrument would need to be built. Although there is not sufficient space to address all issues uncovered by the provincial review, the following epitomize some of the more problematic areas. (For a fuller discussion of the issues surfaced by the provincial review, see Appendix 4.) Fiscal Accountability Not surprisingly, fiscal accountability is among the most politically charged issues facing British Columbia's school districts. The provincial review found considerable variation in the fiscal 102 accountability structures adopted by the different school district ABE programs. Some districts did not reinvest the same percent of ABE dollars into adult learners as other districts. This created inequities across the province with regard to access, retention, and quality assurance. The provincial review also found many district administrators were unaware of how much ministry funding had been generated by their ABE programs. The administrators received a budget from the board and were expected to run their programs on the basis of that funding, but I had access to the financial statements and saw it was not uncommon for ABE programs to generate significantly more funding than they received to run their programs. However, even if ABE program administrators were aware of funding inequities the hierarchical power dynamic within district level bureaucracies would likely dissuade any attempt to challenge standard practice if these administrators wished to retain their positions. Some districts expected the ABE program to be a revenue generator and appeared to hold ABE to a different accountability standard than the K-12 system, which was not expected to turn a profit or operate on a cost-recovery basis. I was made aware that many school districts adopted the "cash cow" perspective toward ABE programs as the most politically astute, expedient way of balancing their budgets. School boards would continuously be alert to areas of opportunity to scrape the needed finances together. Some money might be taken from ABE, some from Alternate Education; no area was sacred. Every year, school districts engaged in the same scramble to balance their budgets because they had to. In one district it was not enough for the ABE program to break even financially but it was also expected to generate a minimum 20% of its revenue for the board. The cash cow depiction many ABE programs laboured under contributed to feelings of conflict, demoralization, and marginalization amongst staff and administrators within the ABE sector. These dynamics often derived from conflict between the 103 business end of school boards (the secretary treasurer's office) and the educational end (the school superintendent's office). These findings of the Provincial Review were vetted through the ABE Accountability Committee with the suggestion that there needed to be a better process to ensure equitable access for adult learners enrolled in provincial school district programs. This resulted in the Budget Breakdown section of the framework which forces districts to disclose the amount of adult FTE funds allocated to ABE programming (see Budget Breakdown in the ABE accountability framework inserted at the end of this chapter). Federal Funding Allocations to First Nations ABE Programs The provincial review surfaced an oversight with regard to federal funds intended for First Nations ABE learners. At the time of study, federal funding for First Nations ABE students exceeded MoE funding for ABE students by approximately 50%. These monies, identified through the nominal roll (First Nations student registrations) were transferred to the Ministry of Education, and funding for First Nations ABE students were allocated back to the school districts. However, the MoE through an oversight was retaining the difference between what the federal funding allocated ABE learners and what the provincial government allocated, which amounted to about $2000 per FTE. The provincial review's report to the ABE accountability committee advised that these monies be returned to the district to support the adult learners the funds were intended to serve. As a result, embedded in the accountability framework's design is the capacity to gather data on First Nations learners, but the mechanism is of course limited in managing how the province distributes federal funding intended for the First Nations population of ABE learners. Nevertheless, the ability to provide both levels of government with data on 104 First Nations adult learners that includes this funding information could promote more intergovernmental debate and First Nations' awareness with regard to current provincial policy. The rationale for including this potential controversy in the framework was to give the mechanism greater efficacy as a tool to allow districts, if they chose, to bring to the forefront a once largely invisible issue to empower a sector within ABE that was being neglected. Enrolment The provincial review found the absence of standardized reporting practices particularly troublesome when accounting for enrolments. At the time of this study, practice included some districts reporting adult FTE students through a regular school process during the September 1701 count only, while other districts reported adult FTEs through a continuing education (CE) number, allowing adult FTEs to be reported twice, once in the September 1701 count, and once in May. The difference in reporting practice created problems with regard to accounting for fundable adult FTEs. If a district reported all its adults in September there was no mechanism to determine how many of those students left the program prematurely, yet received a full year of funding. This created a "Casper Syndrome", a term I use to define districts that received funding for "ghost" students, then allocated adult FTE funds to other program areas. There was also no consistency in reporting students registered through CE numbers. Some districts that reported an FTE (which comprised a student with eight course registrations) in September did not reduce the FTE status in May of students who subsequently dropped courses or left the program prematurely, once again creating the Casper Syndrome. Such anomalies when revealed by ministry reviews stigmatized all ABE programs as cash cows, when this was not the case. 105 The issue of registration and enrolment in ABE surfaced frequently in the provincial review but never as prominent as when the MoE conducted the external review of school districts that had applied for the reallocation of underutilized FTE funds. One of the districts that requested an external review for additional ABE funding was found to be teaching ESL courses, which are not funded by the ministry, and claiming funding on the basis that the students were part of the district's ABE program. While the external review was not a formal audit but sought to legitimize school district claims for additional funds, a member of the review team represented the Treasury Board's interests and also happened to be a former teacher. On seeing ESL students portrayed as ABE students the Treasury Board representative called for an audit, resulting in a significant ministry claw-back of that district's ABE funds. Other districts were thought to engage in similar practices, but were not visited as they did not request to be assessed for underutilized FTE funds, and have likely divested themselves of the practice after this incident. To achieve a more standardized reporting practice the provincial review recommended that all school district ABE students be reported through a CE number, particularly in the case of continuous intake centres which are possibly the most variable with regard to accountability practices. This meant that every 100 hours the institution would review the student's Individual Education Program, noting progress, attendance, etc. If a student left after attending 200 hours or was released due to poor performance, that information would be passed on to a records clerk who would re-register the student's course count from eight to two. When the May 1701 count was calculated the student formerly enrolled in eight courses as an FTE for the September 1701 count would now be enrolled in two courses, representing .250 of an FTE for the May 1701 count. This was agreed to by the ABE accountability committee and inscribed in the framework's design. 106 Attendance and Progress The provincial review found that imposing a standard for measuring attendance and progress was highly problematic, as the impact of regional demographics and differences in program delivery methods would make these criteria the most difficult to assess uniformly across districts. A large urban district, for example, might have a sizable community of potential learners who preferred programs to be offered traditionally, through a centrally located facility. Students drawn to the program may be highly motivated, completing coursework towards a graduation certificate in a timely manner. A rural district, by contrast, might offer nontraditional programs through a number of satellite learning centres that attract a small portion of highly motivated students as well as a large number of students whose academic success continues to be negatively affected by their previous experience with formal education systems. Their level of educational attainment might be minimal, and their initial indicators of success in education might be most appropriately expressed through changes in self-esteem and social skills. The provincial review found these issues were further complicated by the lack of agreement across districts on how or even whether recording and reporting attendance and progress should be done. In many districts attendance records were kept by a sign-in sheet or entering student numbers into a computer. This often proved problematic as many students would forget to sign in and most districts could not afford to have a gatekeeper enforcing the process. One district did not bother to record attendance, arguing it was an intrusion into adult rights which infantilized the learner, an extreme example, but demonstrating the need for some consistent province-wide understanding of standard operating procedures. 107 Members of the ABE accountability committee understood attendance and progress to be problematic criteria closely aligned with regional differences. They suggested that school districts engage in considerable discussion as to what constituted reasonable attendance and progress for students in their regions. The result would have significant implications in rural areas where students might need to travel long distances to reach ABE facilities but lacked access to public transportation. Infrastructural, cultural, and attitudinal discrepancies across districts meant that some achieved high attendance and progress results due to their (usually urban) location, demographics, and capacity to attract a more motivated learner who could demonstrate greater consistency in attendance and progress. Other districts, particularly in rural areas, often needed to outreach to learners whose attendance and progress was complicated by a number of issues that might have precipitated their initial departure from the traditional educational pathway. The ABE accountability committee agreed to the necessity of recognizing the unique learning needs of adult students across districts and the different delivery methods that districts employed to afford adult learners equity access to a quality education. The committee advised that a profile of ABE learners would therefore need to be incorporated into the accountability processes of the different regions. The committee further suggested that each district have in place a defensible philosophy and process that justified its attendance/progress model. This could include Individual Education Programs, anecdotal progress reports, and other hard copy evidence that clearly demonstrated student performance as it related to the academic and social needs of that particular student and the different circumstances of each district. 108 Admissions The provincial review found that students arriving from other institutions or provinces were often not at the academic level their academic transcripts represented them to be. One district discussed a student who transferred to its program with transcripts that indicated the student had achieved grade 11, but when the student took the Canadian Adult Achievement Test (CAAT), which is commonly used by ABE providers, a grade 5 score was achieved. Another student arrived with a district-issued Adult Dogwood, which was frequently the case prior to the common Dogwood's creation in 1999. This school district had issued a Dogwood to a student who now wished to take Biology 12, but when the CAAT was administered the student scored at the grade 7 level. The school that awarded the Dogwood had received funding for teaching the student through to grade 12, and now the student would have to take further upgrading courses to achieve legitimate grade 12 qualifications to be eligible to proceed to postsecondary training. The student was assessed by the enrolling ABE program to have insufficient background to be successful in a grade 12 course, was reassigned to remedial coursework, and subsequently left the program in frustration. These are two of many inequities reported by the field. The provincial review recommended the implementation of a standard admissions test for all district ABE programs. The results of such a test could be used in conjunction with regional demographics which could include First Nations, ESL, and ethnicity categories to help establish related academic and social baselines for performance measures that would be regionally sensitive. When the issue was vetted with the ABE accountability committee, the CAAT was proposed as the standard admissions test because of its already common usage. It was agreed both colleges and school districts should have the same test. 109 Form, Function, and Purpose As the insert following this chapter shows, the accountability framework that resulted from the consensus-based committee process took the form of an extensive series of questions designed to elicit information on program quality and student progress. The framework was designed to reflect the complex nature of educational accountability, while supporting ministry expectations and enabling districts to demonstrate success in light of the disparate nature of school district ABE programs and with respect to their different contexts. As stated in the framework's preamble, the intent was to gather information on school districts that would give them a means "to describe how their ABE program(s) [were] performing" in terms that could "legitimiz[e] the value of student success within their ABE programs(s) through a variety of performance measures". Districts were to respond as comprehensively as possible to the issues the framework raised prior to a visit from a ministry review team. Designed to collect quantitative and qualitative data, the framework sought information on educational philosophies, goals and objectives; enrolments, admissions and curricula; the district's role in the community; the unique regional challenges it faced; pupil-teacher ratios and teacher qualifications; budgets, technological usage, physical plant; and not least, levels of student empowerment, student services, student socioeconomic profiles, and the criteria used to gauge student success. The query pertaining to success criteria was followed by a number of suggested nontraditional (i.e., non-test-based) measures that were developed by the committee and ranged from discernible changes in responsible behaviours and life styles to exhibiting signs of improved self-esteem and "wise stoicism" (or delayed gratification). 110 A draft of the framework was sent to a select number of urban and rural school districts for their review and feedback during the provincial review. Urban districts were represented by one in the Lower Mainland while three rural school districts selected on the basis of being the most diverse or the most difficult to stereotype were also involved. Al l districts gave the framework supportive feedback. Although these districts did not have the opportunity to pilot the framework, some found that merely discussing what it proposed to do with colleagues prompted stimulating dialogue about educational practice and the concept of legitimizing A B E through nontraditional success indicators. Most districts described the framework as a unique process for demonstrating the successes of A B E students and programs and better understanding educational practice. The challenge of the design was to build a one-size-fits-all framework that the districts could run with to describe their own idiosyncratic problems and legitimize what they did. The issues surfaced in the provincial review demonstrated a number of inconsistencies and inequities between school district A B E providers which became foundational tenets to the evolution of the framework but also animated it by serving a dual role. The issues demonstrated possible problematic or controversial areas of school district ABE accountability which resulted in lenses or filters that when incorporated into the ABE accountability framework enabled school districts to demonstrate how they were or were not accountable. Through these issues, school districts were maneuvered into describing how their A B E programs were accountable, thereby exposing questionable practice. In developing the framework it was discussed as a strategy that would serve to discourage unethical A B E practice; but given that a recurrent theme of the Provincial Review was the uniqueness of each district, the framework's architects also saw the framework as a tool that had the potential to empower school district ABE practice. I l l ' The ABE framework was designed to enable a district to legitimize its practice through a number accountability filters that surfaced through the provincial review as issues of conflict or contention. These issues of contention could then be addressed through a legitimate accountability instrument that enfranchised and empowered ABE students and programs alike through a conduit embedded in formal and informal theory that reflected issues surfaced in the provincial review. These issues were imminent to the design of the framework, intentionally enabling school districts to use the tool to argue for individual difference. The framework would offer a means of decentralizing accountability and provide the basis for building arguments that acknowledged the work school districts did despite a wide range of problematic variables that might be very different in each district. A centralized plan would have reified the learners and may just as well have been patterned after the conventional K-12 system. As a means to empower the ABE practices engaged in by school districts, the framework was constructed on the basis of a philosophy premised on having adult educators struggle with determining what educational accountability was so that they would be forced to ask themselves about the people they were working with. The idea was to push the districts out of their comfort zones and provoke them into thinking that it was over to them to legitimize what they did. No one would do it for them, but they would be afforded "voice" through the framework. The emancipatory intent of the framework was to enable districts to legitimize what they did through explanation. If the districts believed their work had value and worth, the framework asked that they demonstrate to the ministry how their work was effective through explanation that allowed for the expression of individual difference. If the districts believed that what they did was entirely different from other districts because of their particular cultural, economic, or geographic circumstances, the framework encouraged them to build argument to legitimize their 112 differences. In doing so, the framework would give the districts some authority and with that, autonomy and responsibility. To gain ministry support the ABE accountability committee endeavored to create a cost-effective design that was persuasive, tangible, and practical. The overarching theme of the ABE framework was to make it real, make it useful, make it achievable, and make it defensible. The district in question would either be found to be capable or wanting. The idea was to create much more awareness around ABE in school districts so instead of being reified by course completion rates, they would now have made an argument - and the basis of a contract. Building the contract might involve much of the district's staff- ABE teachers, teaching staff from the regular school, school district administrators, and the superintendent. It might take several months to unpack; however, it would bring together a collective voice that would make the contract stronger, more widely encompassing, and more persuasive. The intent of the framework was to motivate school districts to engage in a dialogue process with district senior managers, the school board, the ministry, and the community. The framework would also affect the power dynamics within a school district's internal organization. The superintendent, senior management, and others who might or might not understand the ABE program would need to become better informed because their signatures would be required on the accountability statements submitted to the MoE. Senior managers would need to observe and be aware of what went on in ABE before they acknowledged on writs to the ministry the state of the program in their school districts. Auditors would need to verify ABE program accountability, and in so doing might discover the problematic nature of a standardized understanding of ABE accountability within school district programs and why ABE accountability is so complicated. 113 The ABE framework was designed to have the potential to become an instrument to personalize the education process. It nullified the preference of governments for numbers rather than names. The reification of students makes it easier to cut funding as there is no understanding of student backgrounds and individual differences. Within a traditional audit the process reifies students and precludes student and program uniqueness. This creates a lack of understanding of what is entailed in ABE accountability by countenancing assumptions that uniform accountability standards really exist and can be met. There is comfort and protection to be found in assumptions which the framework would have unsettled by enabling districts to demonstrate that a "one-size-fits-all" understanding of ABE accountability would further marginalize the already marginalized. Implementing the resulting ABE accountability framework would involve mainly time and it would be over to the school district ABE staff to determine how much time they put into it. Some districts might not want to go through this process, but would rather be assessed by course completion rates. Others might come together as a team to legitimize their accountability practice through the framework's indicators of success which they might consider more valuable than course completion rates alone. In this way the framework was intended to give voice to a marginalized sector of the education field as well as to the students that it served. ABE, like other nontraditional educational pathways, has been often perceived as less valuable than traditional pathways, and as with the learners themselves, there is often a feeling of marginalization amongst staff in these programs. The framework provided an instrument to empower this sector. The regular K-12 system has been often profiled; the great things happening in the K-12 system are actively brought to the forefront of the public's awareness on a frequent basis. The same cannot be said of the great things happening in the ABE sector because 114 it does not have a conduit to the public arena. The framework could be that conduit if school districts worked at it. Depending on where the district was located and what population groups it comprised, the framework could pull on special interest groups with political power, such as First Nations groups in some regions, to build a political foundation of support for the framework and the ABE processes conducted by the district. Through community awareness it could engage community leaders in a process that would help them understand more about adult learners and how they could benefit each other through ABE programs. The community needs to understand that the program exists to help learners and society at large. Through the framework school districts could include community voice and wider societal input. Community members with a vested interest in the community's education could be brought in - business groups, the rotary club, religious affiliations, potential students, income assistance officials, local politicians, and others. This process could build argument for decentralizing curricula and accountability structures. One district that reviewed a draft of the framework hoped to use it to introduce a community forum to provide insight into what ABE is, what it does, how it could help community members, and how they could help the program. The framework emphasizes opportunities to personally profile ABE programs: this close, personal vantage point might make it more politically difficult for funding bodies to reify ABE students while allowing them to better understand how school districts are accountable and why they deserve their current levels of funding. Therein lies the potential power dynamic of the framework. It would enable a district's ABE programs to draw attention to what they did, and in the process of building a higher profile, the district could indirectly gain greater community 115 support and political effectiveness. If the framework succeeded at raising the profile of ABE, the program would become far more defensible. It would no longer be something remote and removed that could be milked as a cash cow, but could be argued to be an integral part of the community for a number of learners. Summary This chapter chronicled the evolution of the ABE accountability framework from the fractured data and contentious relationships that marked its origins to its design and ultimate intentions. Initial attempts to design a common accountability standard applicable to both college and school district ABE providers floundered, mainly because inequities in tuition fees, government funding, and student finances failed to establish a common denominator foundational to the proposed framework. The inequities were exacerbated when funding for school district ABE programs was cut and enrolment capped. While some school districts might have turned ABE learners away, others exceeded their cap without getting additional funding. When CEDA initiated talks with the MoE to remove the cap and reallocate funds a number of system-wide irregularities surfaced. It was clear that there was no standard reporting mechanism for FTEs that all school districts followed, no agreement on what constituted a norm or a deviation among school district performances, no consensus on how to measure performance outcomes, and no common accountability structures. Before the cap could be removed it was necessary to at least reach consensus among school districts over what constituted an FTE. This led the Ministry of Education to initiate a Provincial School District Review and other research into the design of an ABE accountability framework applicable to the K-12 system. The 116 MoE's accountability committee set about the task of designing an accountability mechanism that would reflect and respect the complex nature of ABE program delivery, the diversity of adult learners, the disparities of regional contexts, and the multidimensional ways success may be exhibited and measured. The various field studies and research initiated during the eighteen-month period of the provincial review in addition to feedback received from selected school districts emphasized repeatedly that districts were impacted by a number of unique regional characteristics that influenced the delivery of ABE programs and would need to be taken into account. This drove home to the committee that the accountability framework would need to be flexible with regard to regional differences that embodied cultural, demographic, and socioeconomic attributes. The committee agreed the conditions that went into a successful delivery of an ABE program at one place and time were probably unique to that juncture. The framework would need to be flexible, incorporating changes over time as demographics and social mores shifted. The proposed accountability framework was not intended to stand as an inviolable structure but something that would continue to take into account regional disparities, new definitions of accountability, changed circumstances, and different understandings of desirable outcomes. Conceptually, the framework was not advocating standards as a good in themselves, but as the means to achieve fair and equitable practice while valuing the idiosyncrasies that exist from district to district. It argued for a number of standards (such as attendance and progress records) that had surfaced through the provincial review, but within these standards it provided an opportunity for the districts to legitimize their practice. Many of the issues surfaced by the provincial review were a result of district level power dynamics and the purpose of the 117 framework was to expose these dynamics and offer school districts the opportunity to build argument as to why their particular school district was different. Many school district staff would not recognize these power dynamics as issues or problems unless it was pointed out to them. They would not see these as issues at all or understand how they could empower that issue, bring attention to it, make it an argument through a framework that pushed its activities forward as important, valuable, and worthy of funding. The tool that resulted incorporated both conventional and innovative measures of success. Honed and tested through collaborative committee processes informed by research, external experts, and assessments from practitioners in the field, the accountability framework's overall intent was to provide school districts a means to account for the educational outcomes of A B E learners in ways that were both politically expedient and sensitive to the complex nature and development patterns of ABE students. The following chapter analyzes the efficacy of the A B E accountability framework at fulfilling this purpose. 118 Notes The main objective of the survey was to provide a snapshot of adult learners enrolled in school district graduation programs (British Columbia, MoE, 1999c). The survey was conducted by phone and collected data from 1,493 former adult learners who had completed or almost completed their program of studies during the 1997/1998 academic year. The survey cohort was drawn from the 19,728 adult students who registered in September 1997 to take courses in the K-12 system. Of these students, 4,568 were registered in the regular graduation program which was based on 52 credits and required students to write at least one provincial exam; 1,496 students were registered in the adult graduation program which was based on 28 credits and did not require students to write provincial exams; and 13,664 students were below the grade 11 level and not likely to graduate in the 1997/98 school year. The cohort's sample of 5,343 students was limited to the 3,049 registered in BC Tel's database. Of those contacted, 1,493 completed the survey for a response rate of 49%. The sample included an equal number of male and female learners with a median age of 22 (ibid., p. 1). Of those surveyed, 56% were employed with a median monthly wage of $1,560; 58% reported they left school in Grade 12; and only 9% reported leaving school before Grade 9 (p. 2). 2 Sixty school districts received funding for adult basic education; 21 were over their district cap. Of these, only six districts applied through the review process for additional funding. In total, the districts that were reviewed requested an additional 239 FTEs; based on the recommendations of the external review team, 113 FTEs were approved. 3 The First Nations have an Education Achievement Plan which is similar in some ways to the ABE accountability framework in that First Nations representatives meet with the board and bring their voice to the table to express what they want to see happening in their schools. A personalized educational plan that is sensitive to cultural needs within the K-12 rubric is then designed and operationalized for each school district managed by the First Nations. 119 Figure 1* ABE ACCOUNTABILITY FRAMEWORK First Draft August 2000 Developed for School Districts of British Columbia By the Ministry of Education ABE Accountability Committee 1998-2000 * This is the ABE accountability framework as it stood at the completion of its first draft by the British Columbia Ministry of Education ABE Accountability Committee in August 2000. The framework was intended to give school districts of British Columbia the choice to either follow the framework's outline as delineated below or continue to follow existing practices of relying on standardized testing. If the districts chose to adopt the framework, comprehensive socioeconomic data pertinent to the program's client base, registrations, fiscal accountability, and modes of delivery at the district and facility levels that attempted to describe and legitimize the performance of the district's ABE programs would be submitted to the province prior to a visit from a review team to conduct more thorough program assessments. 120 A B E A c c o u n t a b i l i t y F r a m e w o r k The following accountability framework is designed to provide school districts an opportunity to describe how their ABE program(s) are performing so that the districts may legitimize the value of student success within their ABE program(s) through a variety of performance measures. The framework has two main areas: 1. Program assessment (includes budget and socio-economic indicators) 2. Student assessment The school district is to complete the following framework as comprehensively as possible and return it to the Ministry of Education prior to a visit from the MoE's review team. After the team has visited the district additional information may be required. A covering letter from the ministry will provide the district with access to ministry contacts and anticipated timeframes to assist the district in providing this information. 2 121 1 . Program Assessment D I S T R I C T L E V E L 1) Are all fundable adult learners reported through a continuing education number (CE#)? If not, please provide a breakdown of how adult students are reported at the district level. If a percentage of students are reported through a CE# and others through regular/alternate please provide a breakdown. 2) What is the total number of ministry funded FTE adults? 3) What is the total number of fundable adult students (headcount)? 4) What is the total number of feepayer adult students (graduates) in ABE programs? 5) What is the total number of nonfundable adults, i.e., those enrolled in general interest and vocational programs? Please provide a breakdown of enrolment by nonfundable program area, i.e., total number of students enrolled in general interest and total number of students enrolled in each vocational program or diploma area. 6) What is the total number of adult learning centres or schools where adults are receiving instruction? 7) What is the school district's philosophy with regard to the provision of adult basic education? 8) What unique challenges does the district feel impact its role in the provision of ABE? 9) How do you feel the district could more effectively deal with these challenges? 10) Please describe your district's involvement with the regional planning process, whether or not it has been successful, and how it could be improved upon. 11) Please describe partnerships that you have initiated to better serve the needs of learners in your district. These partnerships may or may not be associated with regional planning. 3 122 F A C I L I T Y L E V E L A "facility" could refer to a learning centre, or an ABE program offered as part of an alternate program or through a high school. If the district has adults enrolled in only one centre or school, this process will be simplified accordingly. For each centre where ABE is being offered please complete the following: 1) Is public transportation a problem for some centres? 2) Does the centre provide daycare? If so, is there a cost to students? 3) Describe your centre's role in the provision of ABE and how it is reflective of the district's philosophy regarding program delivery for adult learners. These could be described as goals and objectives for the centre. 4) What unique challenges does the centre feel impact its role in the provision of ABE? 5) How do you feel the centre could more effectively deal with these challenges? 6) Please provide a percentage breakdown of students at each level: • ABE fundamental - below grade 8 level • ABE functional - grades 8 to 10 • ABE provincial - grades 11 and 12 7) What are the centre's long term goals with regard to ABE? 8) Please provide a breakdown of enrolment and resources in each centre/school. • Location/address/phone/contact person • Programs offered at the centre/school • Is it a regular high school - do adults attend classes with regular high school students, or do they have an adult only learning environment? • Is it an alternate school - are adult students included in the same facility with school aged students? • ABE • General interest courses • Locally developed non-fundable certificate/diploma programs • ESL • Other 4 123 F A C I L I T Y L E V E L , c o n t ' d 9) Regarding the ABE program component only, within each centre/school what are the program's: • Months of operation • Days of operation • Hours of operation • When was the ABE program initiated? Month/Year 10) With regard to student tracking in ABE programs, please describe how you track student progress, i.e., is this done through a computer program or manually? Your description should include a discussion regarding how often you monitor a student's progress and how you claim course funding for students as the year progresses. That is, if you claimed a student as an eight course FTE in September and the student did not attend regularly and/or made minimal progress how many courses would you claim for funding in the May 1701 count? Please address the following points in your description of student tracking methods used in this district. • Attendance expectations • Criteria and justification • Anomalies • Remedial and intervention techniques • Student progress • Criteria and justification • Anomalies • Remedial and intervention techniques • Number of FTE teachers • Staff schedule • PTR (Pupil-Teacher-Ratio) and rationale • Counselling for career/personal/academic issues • Number of ministry funded FTE adults enrolled at the centre • Number of fundable adult students enrolled at the centre • Number of FTE students under nineteen years of age 5 124 F A C I L I T Y L E V E L , c o n t ' d 11) Describe the type of facility ABE programs are offered in. Does the district lease the facility? What is the rate? What is the facility's total square footage? 12) What technology is utilized for ABE program delivery? E.g.: • Computers/printers • Photocopier • Audio/visual • Scanners etc. • What courses are offered through technology? Notate courses and grade level offerings; e.g., Math 12 • Course delivery, i.e., self-paced/classroom, Nautikos, other • Locally provided distance education for adult learners 6 125 B U D G E T B R E A K D O W N Please submit a breakdown of the budget for the district's ABE programs. This budget should reflect the allocation of adult FTE monies to support fundable ABE adults only. If a centre is 50% adult and 50% youth then all associated costs for adults would likely be 50% of the total cost. This would also be the case with multiple non-fundable programs being offered through the same facility where ABE programs are housed. The budget should demonstrate how ministry ABE funds are allocated within the districts to support adult enrolment in ABE programs only and ABE program budgets should reflect only those costs associated with supporting the delivery of ABE programs. Example of Budget • District administration cost and justification rationale, i.e., percentage of FTE monies allocated to district administration in support of ABE programs • Administrator costs • Teacher costs • Support staff • Student support workers • Clerical • Janitorial • Lease costs • Technology costs • Travel costs • Professional development costs • Sick time costs • Costs of supplies • Other costs • Total ministry adult basic education FTE dollars/total ABE program costs 7 126 S O C I O E C O N O M I C I N D I C A T O R S The factors listed below are intended to elicit information with regard to the socioeconomic situation and learner demographics of the adult client base you serve. If there are factors not included here that may result in the social exclusion and educational marginalization of adult learners and impact their access, participation, motivation, etc., please include these as well. • Age • Sex • Ethnicity • First Nations • Non-English home language • Income • Income source • Employment history • Marital status • Children • Length of time out of school • Highest grade level achieved • Province or country • Last high school attended • Province or country • Reason(s) for leaving school • Transportation 8 127 2. Student Assessment Each district/centre is unique. Assessment of student success may therefore vary. The following indicators are examples of how each centre may choose to demonstrate student success through the assessment of quantitative and qualitative indicators that may be demonstrated through intended and unintended outcomes. These indicators are not meant to be prescriptive, and do not encompass all possible signifiers of success. They are examples of assessment indicators that a district might use to legitimize the value of student success at each individual centre. They could be arrived at in collaboration with staff, students, community members, and other stakeholders. 1) Student perception of intended and unintended learning experiences. 2) Credential. E.g., • Regular Grade 12 Dogwood, i.e., 13 courses • Adult Dogwood, i.e., 5 courses • GED • Certificate Programs, i.e., Computer, etc. 3) Ratio of course registrations to course completions. This should reflect the number of courses that each student was funded for and how many were successfully completed. 4) Availability of nontraditional educational programs to augment the traditional K-12 curriculum (possibly offered as certificate programs): • Career Prep, Co-Op, Apprenticeship, Resume Writing • Specialty non-academic (K-12) curriculum, i.e., Superhost, Firsthost, Healthy Children Healthy Futures, First Aid, Foodsafe 5) Transition to: • Further Education • Employment 6) Responsibility to self and institution: • Phones in when away • Works diligently • Attendance • Progress 9 128 2. Student Assessment, cont'd 7) Internal changes or outcomes in sense of self indicators 8) Entrance/exit evidence of improvement in the following areas: • Ability to communicate; speak, read, write, listen • Discourse, Portfolios, Exhibits • Problem solving • Numeracy • Tests, Portfolios, Projects, Exhibits • Measurement and counting on the job • Counting money and change • Counting calories • Paying bills • Balancing check book • Filling out money orders • Helping with children's schoolwork • Filling out Job reports • Use of technology, e.g., knows how to use the Internet for job search • Work habits, e.g., takes pride in assignments • Collaboration, e.g., demonstrates ability to work in team setting • Responsibility, e.g., student takes ownership of his/her learning plan • Community involvement, e.g., knows how to make use of community programs; shows more active involvement in schools and other community institutions • Family involvement, e.g., understands how to access counseling; is better able help children in schooling • Life style, e.g., understands dietary needs) 10 129 2. Student Assessment, cont'd 8) Entrance/exit evidence of improvement (cont'd) • Sense of self, i.e., demonstrates internal changes and outcomes. Sense of self could be expressed through changes in self as related to ABE and expressed through new knowledge and attitudes and satisfaction with accomplishments. It could also reflect social changes such as communication and socialization which could enhance voice. E.g. Self-efficacy: A person's belief in his/her ability to achieve and control actions through: • Choice of behavior: increase in incidence of options towards what the individual feels is most efficacious • Motivation: increase in eagerness to mobilize more effort • Facilitative thought patterns: success scenarios where persons confronted with new learning challenges try to solve the problem and not give up • Perseverance and less vulnerability to stress and depression: length of stay in program Resilience: A product of self-esteem, self-efficacy, and a stable sense of one's self wherein the individual conveys both the capacity to be bent without breaking and the capacity, once bent, to spring back. Resilience is influenced by: Risk factors: poverty, unemployment, race & class inequality, poor education Socioeconomic factors, demographics (age, previous education, ethnicity, marital status, children, employment history etc.) Protective factors: good parents, health, meaningful employment, social supports Luck and timing/context of events Attributional style: optimistic or pessimistic attitude that individuals take towards the good and bad that happen mature ego defenses: unconscious strategies by which our minds cope with stress and danger expressed through: altruism: helping others Anticipation: realistic and effect laden planning for future discomfort to enable greater control over life circumstances Suppression or wise stoicism: ability to postpone an immediate impulse and keep a distant goal in mind while waiting for its achievement Humor Life satisfaction: demonstrated through life story and expressions of purpose for enrolling in ABE. 11 130 Chapter Five The Efficacy of the ABE Accountability Framework: A Critical Assessment The ABE accountability framework was designed to elicit considerable amounts of quantitative data and assist school districts with recognizing the qualitative dimensions of their adult basic education program areas. As the previous chapter noted, the framework was developed in response to findings of the Provincial School District Review. Although the framework did not address all the issues disclosed by the provincial review, the consensus of those who reviewed the mechanism was that it was adequate for field testing. It was anticipated that information gleaned from the pilot trials would provide the framework with the necessary refinements to better ensure its clarity, comprehensiveness, and capacity to meet the needs of the ministry and the field. The initial intention of the ABE accountability committee was to systematically test the framework's efficacy by having five districts in the province (representing the Central, Interior, Northern, Island, and Lower Mainland regions) pilot the mechanism over a four-to-six-month period. During this period, each school district would respond to relevant questions in the framework as comprehensively as possible and then submit their reports to the ministry. The districts would then be visited by an external team made up of ministry and field members, all having adult education backgrounds. After the external review team's reports were submitted, the directors or delegates from the representative districts would take part in a roundtable discussion with the ministry to address issues and logistics of the framework's design and reporting procedures. The perspectives of the British Columbia Teachers' Federation would then need to be addressed. 131 To properly test pilot the framework, fine-tune its format, and build consensus among significant players for its implementation could therefore legitimately take three years or more. However, due to a change in government, the formal pilot process succumbed to an informal school district review. Instead of being piloted in five districts over approximately five months, the framework was piloted in one large rural school district over a five-week period. Considering the effort, time, and resources invested in the framework's design, the reduced scale and duration for evaluating the framework's effectiveness was deemed by the MoE accountability committee to be better than letting it die with no test piloting at all. This chapter describes the informal review process used to evaluate the framework's effectiveness at addressing the more salient accountability issues raised by the Provincial School District Review. Focus is on the framework's capacity to identify and nurture traditional and nontraditional indicators of student success in light of insights gained by the review members' experiences in examining the framework. Based on feedback provided by the review participants, I provide a critical assessment of the framework's constraints and limitations. The analysis is informed by my initial assumption when beginning the study that for a change in process imposed from outside onto established professional practices to succeed would require that the members affected buy into the concept and make it their own. I assumed this to be the case particularly when a politically charged concept like accountability was pressed upon a profession as politically aware as teaching. My critical assessment of the framework's viability and problematic areas is derived from informal discussion with a group of volunteers who reviewed the efficacy of the framework through their professional values and the idiosyncrasies associated with their school district ABE program. 132 Reviewing the Framework The school district review was conducted in the Spring of 2002 at a large rural school district that was among the most diverse in terms of adult learners. The district had a central adult learning centre and a number of smaller satellite adult learning centres in neighbouring communities. The district's adult education department offered the provincial Adult Graduation Certificate (Adult Dogwood) and the regular Graduation Certificate (Dogwood). The framework was reviewed at the largest centre which enrolled nearly 600 adult learners with a large proportion (approximately 25%) being First Nations. The framework was reviewed informally by a group of the school district's ABE educators over a five-week period. The group comprised six educators (five teachers and one senior administrator) who were involved in the district's delivery of ABE programs. These people met once a week after school for at least an hour, sometimes two. All had volunteered to engage in the process on their own time. In between the weekly meetings the group members were engaged in reflecting on the framework's questions, and many communicated their issues and concerns with me on a frequent, sometimes daily, basis. Because of travel considerations, participation was confined to those working at the main centre. From a total of twelve ABE teachers employed at the centre, five participated in the review, representing a sample of 42% of perspectives from that centre's teaching professionals. Although the framework review process could easily have accommodated student voice, the district did not include students as the participants felt that the five-week process was too abbreviated. The rushed approach taken to review the framework was a crucial weakness in determining the framework's efficacy. The staff members who participated in the five-week school district 133 review were frustrated with the abbreviated timeframe. Through discussions with participants involved in the process, I understood there was considerable confusion about the task at hand during the early sessions. The review members initially thought that they were to complete and implement the entire ABE accountability framework by gathering data on program and student assessments. The participants received minimal direction, but were rather left to grapple with the meaning of accountability on their own terms and apply the framework to student assessment in ways that reflected the values they brought to the education profession. They had not been informed that this was the intention of the ABE accountability framework's architects. The goal of the ministry's accountability committee was for the educators to realize they could "own" the accountability process so that it served rather than restricted educational practice. While the participants eventually communicated an understanding of this aspect of the framework's potential, it was apparent through their frustration that the ABE accountability committee's original notion of a four-to-six-month formal pilot project, not a five-week informal review, would have resulted in a less confounded and more informed demonstration of the framework's efficacy. However, even in its abbreviated form the framework did, through the district committee members, elicit a profound understanding of educational accountability as it related to student assessment in an ABE program. The framework has two essential components, the program assessment and the student assessment. The program assessment component is technically oriented and best understood as an administrative function. As can be seen in the framework (at the end of chapter four), the program assessment portion has two categories of questions addressing ABE programming at the district and facility levels. Two further aspects of program assessment gathered financial and socioeconomic information. It was explained to the participants that the administrative functions 134 of the framework were not relevant to the review process due to the abbreviated timeframe. The focus of the teachers then, shifted to student assessment, while the ABE administrator took responsibility for reviewing the program assessment component of the framework. Although the administrator took part in piloting the student assessment component, this part was primarily led and directed by the teachers. We now turn to address the framework's efficacy in dealing with the issues involved in program and student assessments. Program Assessment If the framework had been adopted, the school districts would have the option to either follow the framework's outline (as presented at the end of Chapter Four) and provide the ministry, as comprehensively as possible, with a variety of financial and socioeconomic data that described and legitimized the performance of their ABE programs, or follow existing practice of relying on standardized testing. If the districts chose to adopt the framework, program assessment data would be submitted to the province prior to a visit from a review team to conduct more thorough program assessments. When this district tested the framework's capacity to meet its intended purposes, many administrative aspects demonstrated high levels of accountability and will not be addressed further here. They were viewed as self-evident technocratic/administrative functions and were briefly introduced in Chapter 4 (under the "Informing Elements" section) and are summarized in Appendix 4. They arose as a result of the provincial review and were embedded in the framework's design to gather data on issues that districts had previously reported on. 135 The limited timeframe prevented the school district review from focusing further on the framework's efficacy at gathering data. The framework's capacity to deal equitably with the program issues that surfaced in the provincial review were examined by the district's administrator, were viewed as appropriate standard operating procedures, and where relevant, this perspective is included in Appendix 4. We now turn to the focus of the school district review process which was on examining the efficacy of the framework with regard to student assessment and unpacking educational accountability as it related to the district participants' education profession. Student Assessment The student assessment component of the accountability framework was designed to act as a catalyst to help determine appropriate quantifiable and qualitative data that would best indicate student success. Given the abbreviated timeframe, this aspect of the framework became the focus of the school district review process and was integral to assessing the efficacy of the framework. The architects of the framework were particularly interested in how, with minimal guidance, a group of teaching professionals would unpack the concept of educational accountability to determine the framework's effectiveness at identifying traditional and nontraditional indicators of student success. Although the administrator was part of the discussions, this aspect of reviewing the accountability framework was guided through communicative action amongst the teacher participants, with the teachers assuming the leadership role in these discussions and the administrator acting more as an observer than participant. 136 Toward Emancipatory Accountability Through discussion it became apparent that the educators were engaged in a process of trying to better understand qualitative success indicators. A central part of this process involved participants deconstructing their notions of accountability as these related to their intellectual and moral understandings of education in general and the purposes of adult basic education specifically. A dialogue process was discussed as foundational to taking apart long-held appraisals of what accountability might mean and how and why it might effectively be put into practice. One of the goals of the framework, as discussed by the ABE Accountability Committee, was to encourage participants to question their habitual thought patterns, deconstruct the traditional meanings they might have ascribed to accountability, and then rebuild the concept in alignment with their own value systems, moral practices, tacit knowledge, and experiences as ABE educators. Through discussion I sensed an initial confusion with regard to the argument for recognition of qualitative data in the design and implementation of an ABE accountability framework. I perceived this was due to the process having not included the standardized test-driven assessment tools of traditional approaches to educational accountability. However, it appeared that the review participants' struggle to think "out of the box" with regard to accountability was induced by social regularities (Scheurich, 1994) that subversively dictated their understanding of what denoted learner success in ABE programs. The social regularities I thought were influencing their understanding and interpretation of educational accountability were professionalization and governmental ity (ibid.). At least initially, as I listened to their discussion I inferred that these regularities might have undermined the ability of the review participants to understand and accept nontraditional assessment structures as legitimate success indicators for 137 an ABE accountability framework. These social regularities tend to regenerate traditional assessment practices which may act as invisible influences that facilitate the replication of the dominant social order by shaping intellectual and moral activities. They serve to define a predetermined value of knowledge that contributes to the control of social reality. Arguably, the influence of social regularities is not necessarily negative. They can and do contribute to a moral social economy. However, their possible hegemonic influence needs to be recognized to ensure that policy with regard to ABE accountability does not act as a marginalizing agent actively contributing to social exclusion. Professionalization Professionalization is a process of socializing professionals, in this case K-12 teachers, which is initiated with their training and which becomes increasingly embedded into the world view, moral outlook, and personal ethos of these individuals through the course of their everyday work activities. Through indoctrination, educational policy such as standardized testing has become the accepted assessment practice in the K-12 system. This acceptance and reinforcement by professionals legitimizes and ordains structures that determine prescribed notions of what is good and worthwhile as regards a productive citizen (Scheurich, 1994). As the review process progressed, it was evident through discussion that dialogue and debate over curriculum and ministry-determined learning outcomes had intensified. The debate centred on ways of making the curriculum more relevant to the needs of adults who had a different life story than school-aged learners. As the review participants struggled with the curriculum debacle, I sensed through their stories that they were also struggling with years of being professionally socialized into the role of K-12 structures. My impression was that the 138 participants appeared without exception to be uncomfortable with thinking of novel approaches to structuring accountability in the interests of themselves and the adult students. Much of the discussion about accountability within the curriculum seemed linked to professionalization as it was difficult for the review members to "unlearn" the structures they were so familiar with as a result of their experience with the traditional system. Through discussion, the review process participants began to cultivate an emancipatory air by dint of unpacking all that might be implied by accountability and the various purposes adult basic education might serve. It appeared through discussion that the review members were beginning to deconstruct their understanding of learning, success, course construction, and curriculum IRPs (integrated resource packages that contain ministry prescribed learning outcomes). In this sense, the review process was liberating, powerful, and inextricably linked with undermining years of professional socialization resulting from the conventional acculturation of teachers. I interpreted this process as working toward an accountability framework guided by praxis and taking the initial steps toward what I define as emancipatory accountability. It would seem that the review process promoted a deeply reflexive approach to practice, which is rare as practitioners in the field seldom unpack and interpret their day-to-day educational activities but rather attend more to the impacts of policy and other external influences on education in their discussions. Governmental ity Governmentality, as Scheurich (1994) described it, was another social regularity that appeared to have influenced the review participants' discussions. Even when review members got through their own professionally induced limitations concerning accountability, it was still difficult for 139 them to arrive at nontraditional success indicators, largely because they felt that if they did, the ministry would not recognize them. Traditionally accepted assessment mechanisms for educational accountability are most often one-size-fits-all quantitative indicators of academic and social ability that prescribe success through predetermined standards. If you achieve the standard you are successful; if you do not you are a failure. These predetermined standards and values are reinforced through traditional assessment structures and are a form of governmental ity that regulates and stabilizes the social order. The initial difficulty the members of the review process had in arriving at nontraditional success indicators was likely due to their frustration with possible non-acceptance by the ministry. This is indicative of the influence governmental ity works upon teachers to perpetuate prescriptive behaviours that predetermine socially acceptable notions of wellbeing, happiness, and productivity. The struggle with accountability also seemed influenced by an apparent fear of retaliation that the identified nontraditional indicators would somehow not meet ministry conventions and expectations. Although subtle, discussions indicated that a fear of retaliation was present for much of the process. It evidenced specifically in a conversation where review members discussed the possibility of outcomes not complying to convention. This fear complex I felt was influenced by governmentality. Thus it was understandable that the review participants approached the prospect of evaluating the framework's efficacy in what they relayed through conversation as a "negative" frame of mind, "uninspired" by an activity they deemed "ineffective at best". Nevertheless, despite the low expectations, nihilism, and culture of defeatism emanating from the early stages of the review process, which would seem to militate against any conceivable form of innovation, the process eventually achieved remarkably creative results in a limited five-week timeframe. 140 Voice and Power An initial breakthrough was made in discussions concerning the liberating qualities of adult basic education. It was apparent the review members felt that for many adult learners education was an emancipatory process that could help students better understand and alter the conditions that prevented them from living more fulfilling lives. Voice and power were discussed first as these were seen as foundational emancipatory concepts that acted as the basis for an adult learner's success but could not easily be demonstrated through quantitative analysis. Thus it seemed the emancipatory implications of ABE could legitimize accountability structures of success indicators that included voice and power. Core Course Development The review members examined the possibility of developing courses and units that could demonstrate fundamental skill sets and become aspects of conventional academic course completions. These programs would be classified as "fundamental" ABE course completions in the sense that they would be necessary for students to move on to academic courses, lifelong learning, and employability. These requisite courses or units and skill sets would be course/certificate completions predicated on indicators of student success for learners who experienced difficulty with academics. By recognizing these skills through course/certificate completions both the student and the program could provide funding bodies with a more accurate and relevant demonstration of how they were successful. Review participants favoured different though complementary changes to course modules. One argument advocated the possible implementation of a stand-alone fundamental course such as a version of the Career and Personal Planning (CAPPA 12) unit which could assist students with 141 academic achievement by focusing on organizational skills that might include a number of subset attributes such as self-efficacy, resiliency, taking responsibility, etc. Another suggestion was to develop a version of Communications 12 that from a curriculum development perspective would be andragogically aligned and recognize and value a student's life story. Outcomes would be related to the real world and could include employability skills such as resume writing and other forms of practical writing, effective oral communications, listening skills, etc. Another idea was to augment the existing higher level academic curriculum with employability skills. Through splitting the curriculum into smaller "mini courses," it would arguably be possible to create curriculum that better fitted the needs of students and employers as well as met the IRPs. This could help students become more successful and provide greater flexibility in how teachers delivered curriculum. Yet another recommendation was to isolate and recognize nonacademic success indicators fundamental to student academic success in the broad ABE curriculum. Students demonstrating these nonacademic indicators would receive recognition of success, which would possibly boost the student's self-esteem, confidence, and motivation to work toward academic success indicators. This suggestion received the group's wide support and led the review members to work toward identifying nontraditional indicators of student success. Student Diversity Through discussion, it appeared that review participants had become focused on trying to develop and legitimize success indicators for students who had a difficult time with academics, were often marginalized and socially excluded, and whose progress was the most difficult to demonstrate. It was indicated through discussion that some students had made giant leaps forward in terms of their self-esteem and social skills. These were discussed as examples of 142 nonacademic outcomes that had undoubtedly contributed to student success in academic areas, and the recognition and valuing of these outcomes were critical to legitimizing both program and student success. There was also discussion about the possibility of incorporating these and other outcomes into existing curriculum so that students were recognized for success in nonacademic areas which would be good for the success rates of both students and programs. Much of the discussion focused on students whose success was not easily demonstrated through traditional accountability structures such as test scores and course completions. Yet these students attended consistently and worked hard. As the discussion evolved, the issues of particular students were focused upon. These individuals had significant learning problems and would likely never make great academic gains. However, they had made outstanding progress in a number of other areas. These included self-esteem, responsibility, resiliency, organizational skills, attitudes toward schooling, and working on school work at home which provided positive models for the children living in the same quarters. The review participants cited the instance of a student who had been in school for two years. The student had not yet completed a grade 11 course but had completed several levels of literacy and numeracy. The student's attendance was good, progress was steady, and the ability to fulfill the graduation requirements had been demonstrated. However, due to the student's lack of progress as determined through grade 11 and 12 academic course completions, the sponsoring agency (social assistance) chose to look for a quick-fix, privately-provided work program which it hoped would get the student off income assistance. But this would not help the individual achieve grade 12. To remain on income assistance the student was forced to leave ABE for a six-week work program, which unfortunately did not lead to secure employment. The student eventually 143 returned to the ABE program the following school year, but in that absence much ground had been lost in the student's numeracy and literacy skills and the student's self-confidence had deteriorated. This student was not an isolate. As the review participants probed deeper, they noted a number of skill sets that were paramount to individuals making academic gains but were not recognized through conventional course completions. These skills often legitimately took the learners several years of hard work to acquire, and needed to be valued if governments were truly interested in a long-term policy solution for students enrolled in ABE programs. Nontraditional Indicators of Student Success Participants indicated that their discussions had begun to focus on success indicators designed not to control performances but to enable student transformation in attitudes toward learning, personal behaviours, sense of self in the world, and orientations toward society. These discussions led to the realization of nontraditional indicators of student progress which could, in combination with the quantitative aspects of the framework, demonstrate program accountability. The review participants felt these were best described as nonacademic learning outcomes contextualized to the learning environment. In no particular order, Table 5.1 summarizes the nontraditional indicators the review participants agreed were important to their specific school district. 144 TABLE 5.1 NONACADEMIC LEARNING OUTCOMES CONTEXTUALIZED TO O N E LEARNING ENVIRONMENT ABILITY TO • ask for help and seek clarification • work independently • solve problems • learn and adapt INCREASED • interest in new ideas • flexibility • motivation • responsibility • self-confidence • assertiveness • persistence DEVELOPMENT OF • effective social skills • leadership abilities • more positive personal attributes • a goal orientation • positive lifestyle changes Source: Fieldnotes documenting impressions of School District Review Members. These indicators of student success are oversimplifications of more complex attributes that are not clear-cut but are rather intricately entangled. Each of the short-listed attributes mentioned in the table is a composite of other desirable qualities. The ability to ask for help and seek clarification, for example, is dependent on the development of increased self-confidence and assertiveness, motivation, persistence, social skills, and other characteristics ABE students often find challenging. The ability to work independently embodies the capacity to organize oneself and one's materials, prioritize tasks, set and meet deadlines, follow directions, and locate additional study resources if needed. Problem solving implies a readiness to devise strategies for attacking a question, to make decisions, and think of contingency plans, which requires resourcefulness, the ability to think creatively and critically, and the capacity to link seemingly disparate concepts in integrative ways that have practical applications. Having the aptitude to 145 learn and adapt denotes not only the motivation to acquire new skills but the capacity to transfer one's knowledge and understanding to new situations. In some ways, intellectual curiosity suggests a commitment to, and enthusiasm for, lifelong learning. To demonstrate flexibility intimates a capacity to handle changed circumstances, deal with uncertainty, and be open to new ideas. Motivation embodies self-confidence, a positive attitude, a goal orientation, determination, self-mastery, and a commitment to learning as a means to achieve. Responsibility includes a host of attributes such as reliability, consistency, commitment, punctuality, and seeing things through. The development of effective social skills is a complex strata of attributes that might involve some capacity to feel at ease in social situations, work in groups, understand other cultures, and exhibit empathy, compassion, tact, politeness, respectfulness, humility, honesty, and helpfulness. Leadership abilities are social skills of a higher order which the review participants suggested could be demonstrated by a student becoming a role model for others, engaging in the education process at school, encouraging others to attend, and modeling positive study practices at home. Personal growth is another complex of characteristics susceptible to transformation through education. One personal change that affects other changes, particularly voice and power, is the development of self-confidence and assertiveness. Self-confidence manifests in striking and subtle ways, in a student's willingness to exercise voice in advocating for her/himself as well as for others, or in moving from a state of fear to an understanding of oneself as a competent individual, or in some cases in being able to hold one's head up, make eye contact (when this is not a cultural issue) and keep one's posture erect when speaking with others. It impacts upon the development of attributes essential for student success, such as persistence (which implies patience, a futurist orientation, a belief that one's efforts will eventually meet positive results, 146 and a goal orientation, which in turn involves planning, strategizing, and learning to trust others whose assistance may be necessary to help reach one's goals). The review participants pointed to other personal development characteristics that lay the basis for student success, including the ability to take positive criticism, laugh at oneself, remain calm in situations involving frustration, cope under pressure, and manage anger, all of which ABE programs might influence. The review members thought positive lifestyle changes might also be indicative of student progress and program success, and could be demonstrated through a student's willingness to delay gratification, quit engaging in substance abuse, make positive changes in personal appearance, gain employment even though a graduation pathway might be incomplete, and overcome poverty and other lifestyle factors which might have held the individual back in the past. All of this would indicate initiative on the part of the student and a decision to take personal control over one's life and assume positive patterns of behaviour. In common with a host of theorists (Beder and Valentine, 1990; Boudreau, 1998; Cross, 1981; Ranson, 2000; and Upex, 1999), this gathering of practitioners understood that the acquisition of self-confidence, self-esteem, and other nonacademic outcomes were the necessary bedrock for any academic success. In addition, these attributes might contribute to helping students understand the importance of exercising voice, which would better enable them to make positive changes in their social sphere. In this way the accountability framework could become an educational policy argument for beginning to offer long-term logical solutions to problems of social exclusion which is the basis for many pathologies in contemporary Canadian society. 147 Devising Alternative Pathways to Graduation As the review process moved forward, it seemed the discussions had clearly helped the members to begin thinking outside traditional structures. This was indicative in their proposal that three graduation pathways be instituted to reflect the students' different degrees of academic and nonacademic accomplishments. One pathway would lead to the Adult Dogwood and could be based on coursework only or coursework combined with a certificate program such as Web Design. Another pathway would culminate in the GED (General Educational Development) Certificate, accepted by many businesses as a grade 12 equivalency. This route is an expedited credential pathway that is far less rigorous than the traditional grade 12 certificate yet demonstrates a recognized level of academic achievement. This second pathway would be for students who only wanted a GED or for those who found the grade 11 and 12 academic requirements too rigorous. A third pathway would terminate in a School Leaving Certificate that had an employability skills focus. This would be for students who could not achieve a grade 10 level and would be designed to facilitate their entry into the workforce. These pathways in combination with nonacademic outcomes were valuable inroads with regard to arriving at nontraditional indicators of student success that could be employed to legitimize the success of the program. Software Coding of Qualitative Indicators As a result of the review participants' initial work on accountability, and in particular their efforts at defining nonacademic outcomes, the program administrator had one of the members examine the applicability of computer software for coding and recording qualitative success indicators. The review members proposed to look at the possibility of developing indicators for 148 the nonacademic outcomes using QSR-NUD*IST 6 as a code-based qualitative analysis platform to make sense of the non-numerical database synthesized from the nonacademic outcome data. Once the data was codified the district would use SPSS (Statistical Program for the Social Sciences) to further quantify the data. The district, in conjunction with CEDA and the ministry, planned to attempt reaching a common understanding and acceptance of this assessment of nonacademic outcomes of student success which it saw as foundational to future academic success, lifelong learning, and employability. The qualitative and quantitative data analysis programs would have added credibility to the inclusion of these indicators in the accountability framework as valid signs of success. This argument was presented by the president of CEDA (myself) to the Deputy Minister for consideration, but was shelved with the change in government. The Review Process The review members' engagement with the concept of accountability was a delicate process highly susceptible to change from outside influences. At one point, the participants met with the addition of an external observer, a director from another BC school district. Through discussion it was evident that some members were eager to share results with this individual, while others were more reserved. I understood that there was initially a more reluctant than usual engagement in discussing the many ideas that previously arose when unpacking accountability. The external participant's visit seemed to provide further evidence of how power structures limit our thinking about accountability. 149 In addition to a school district director, a ministry official was invited to observe the review process. The ministry member could not attend, however. It was later pointed out by the external school district participant that it was perhaps better the ministry official could not be there as the former was very aware through body language and the initially subdued atmosphere that the school district review members were reluctant to discuss their results, most likely because their ideas were not aligned with conventional notions of accountability. I inferred that their discomfort most likely stemmed from the ominous connotations associated with accountability, especially for the education profession. The term seemed to immediately conjure up a punitive form of measurement that was too often predicated on reification and standardization. I understood that members of the review process seemed to feel that either they or the program were being measured or assessed and they were trying very hard to ensure they said the right things. Their guardedness diminished over time, but it might not have if two external members had been brought into the discussions, either separately or together, particularly in light of the short duration allowed for the assessment of the accountability framework. At the end of the review process the participants communicated their thoughts on the success or failure of the framework's capacity to promote dialogue on the meanings of accountability and how the concept might become an asset to educational practice. Although there was some frustration regarding their perceived lack of a concrete structure, overall they suggested their involvement in the process was very valuable. They stated the process was successful, with qualifications. Many noted that prior to the process, issues around accountability were not being addressed at all, and without the process it was doubtful they ever would have been. Some members appreciated that the process gave them an opportunity to formally reflect upon and explore the apparent banalities, intricacies, and consequences of their daily professional practice. 150 Others noted the process had encouraged thinking out of the K-12 structures, and these out-of-the-box imaginings were becoming habit forming. The thinking exercises promoted, some suggested, a deeper understanding of why adult learners might find it difficult to stay engaged with formal learning structures and make progress. Some members came to the conclusion that many courses offered to adult students were unnecessary. The process led a few members to think about a number of innovations that could be made in the curriculum to better serve the needs of adult learners. The success of the review process was further validated by the external observer, a director responsible for ABE programs at another school district, who commented that the process appeared to have prompted the review members to take a hard look at the quality of their adult basic education programs and begin discussing the future direction of their particular organization, and debate what factors determined student success. This observer substantiated my informal discussions with review participants. Several of those critiquing the framework noted the review process provoked thinking of better ways to identify the successes for which ABE programs could take credit. It provided a forum, some members contended, to validate the activities educators engaged in with students that were not strictly academic but were as valuable in terms of providing students the skills and motivation to become lifelong learners and productive members of society. Such activities, the review members agreed, often went unrecognized when programs and students were evaluated only by quantitative measures. The participants also referred to the review process as laying the groundwork for determining future directions in ABE programming. Many members lamented the short duration of the review process and advocated a more in-depth exploration of the possible meanings and directions of accountability. Some felt an emergent sense of ownership and responsibility over how accountability was defined, but as one member noted, this sense 151 dissipated upon realizing the individual was "part of an experiment". What was needed for the participants to truly own and realize accountability, this individual advised, was "more conversation, brainstorming, possible scenarios, and assurances that the end product will be better." More communicative action might indeed have resulted in a tangible product: many committee members expressed disappointment with the process for not resulting in the design and implementation of an instrument that would allow them to demonstrate how learners were successful as a result of attending the district's ABE programs. The task, however, as I understood it, was not necessarily to come up with a product, but to ascertain the value of the framework as a mechanism for determining ABE accountability. The results of the review process were presented in a report to the ministry. At the time the province's policy climate was highly unpredictable. The review had been conducted during the first year of a newly elected Liberal government and it was highly uncertain which policies, if any, of the former NDP government would be adopted by the new regime. With the change in political administration, there was a change in political priorities. The rights of the poor and undereducated did not factor high in the new government. The report's findings and the ABE accountability framework as a whole were shelved without further discussion. The main drawback to the framework's review process and possible implementation was the change in government. Prior to the change in government, the directions and momentum coming from the ministry were positive. It was thought by the ABE accountability committee that upon completion of a draft accountability framework, approximately three years would be required to properly pilot the framework, fine-tune the mechanism, build stakeholder consensus, and implement the test results as ABE accountability policy for the province of British Columbia. 152 The five-week review was an attempt by the MoE Accountability Committee to salvage close to three years of work that had gone into the framework's development and ensure the work had some meaning. The results of the district review tended to reinforce my belief that the framework's design was able, even under substandard circumstances, to exhibit a positive force. A central flaw to the framework is that it is long, involved, and labourious. The entire process takes work because a district is trying to legitimize programs for learners who were not successful in traditional structures. The battle waged against convention is entirely uphill. Nevertheless, from the feedback I received from the review participants it would appear that although the process was extremely labourious for them over the five-week period, their main complaint was that they did not have the time to do it properly. A further drawback with the school district review of the framework was that it took place in May and June, which is the least opportune time of the year to engage teachers as they tend to be too tired to take on extra commitments. The original intent of the MoE accountability committee was to initiate the process in October, once students were registered and classes had begun. The district could then pace the process, breaking it up into do-able chunks over the course of a year which would give people enough time to reflect on it. The ABE framework review process is complicated, and my guess is that many ABE program providers would choose not to do it, especially in urban areas where it is less of a community program. However, for districts that wanted to make the effort it could become a means to empower school districts and learning communities within those districts. Even the substandard five-week school district review demonstrated the framework's ability to effect change. 153 Although the major complaint of the participants was that the process was too short, they indicated that their frustration was that the process had just gotten started, they felt it could have gone further, achieved more, and they wanted to continue the process to see if something real and tangible could be produced. Summary This chapter assessed the efficacy of the MoE accountability framework. Through the program assessment the ABE framework appears to have been a potent tool for gathering data on the issues surfaced in the provincial review of school district ABE programs. It could provide the ministry with a means to develop policy on a number of issues that could encourage a much more standardized and accountable process for the delivery of ABE programs in the province's school districts. The focus of this chapter, however, was on the school district review of the student assessment aspect of the framework. The objective was to determine whether the mechanism could offer school districts an opportunity to legitimize the success of their ABE programs. While not discounting quantitative data, the MoE accountability committee anticipated that the framework might provide an opportunity for the districts to make argument for other indicators of success of a more qualitative and nontraditional nature. The review district arrived at a number of qualitative indicators related to student success that were not limited to standards or course completion rates and it put forward a means of codifying and analyzing data that might lend more credibility to nontraditional indicators of success. This combination of quantifiable and qualitative data gathering and analysis processes would provide an opportunity to validate nontraditional outcomes of educational accountability and to address 154 the multifaceted needs of adult learners. In this way the framework would support learners through an accountability process that was predicated on inclusion and would not act as an agent that promoted social exclusion. This validation of nontraditional indicators of student success would provide a greater opportunity for each district to offer educational programs that reflected the demographic needs of its client base. How each district might determine which nontraditional success indicators best reflected the socioeconomic and demographic conditions of its adult students was a complex aspect of the framework. Through the framework review, and subsequent discussion, I surmised that the influence of professionalization and governmental ity had shaped the participants' views of education, their understanding of the purpose of schooling, their conception of society and the curriculum, and the educational tasks and activities they considered important. Traditional assessment has been an educational activity that defines student performance through a predetermined lens. It has been a difficult habit to break. But it was not impossible as evidenced by the many nontraditional indicators of success the review participants surfaced during the process and the initiative they took in devising a model of alternative pathways for students to achieve graduation. These were highly significant outcomes of the piloting exercise. Another result, exceptional because it was wished-for without expectation of being realized, was seeing that the framework could act as a catalyst to encourage debate about practice and radicalize the education professions' thoughts on the value of accountability. The review process lent support to the other school districts' favourable perceptions of the communicative impact embedded in the accountability framework. By informally reviewing the 155 framework, the framework's capacity to function effectively as part of the everyday moral practices of educators was revealed to be closely linked to a process based on discourse, in this instance discourse about what constituted success among ABE learners. The process proved to be inseparable from determining the strength, flexibility, and viability of the framework. The assessment of the framework demonstrated its ability to gather technical data on each district's ABE program. This was an administrative function, which involved little time and addressed issues surfaced in the Provincial Review that the MoE wanted more information on. This data could be used by the ministry to develop policy ensuring greater equity within school district ABE programs. This would provide the student with greater quality assurance and the ministry with greater transparency regarding ABE funding. The student aspect of the framework demonstrated that even through an abbreviated five-week school district review process the participants were able through communicative action to break with educational accountability conventions and self-censorship behaviours that were likely influenced by professionalization and governmentality (Scheurich, 1994). In many ways their struggle was with the development of success indicators associated with what Ben Jaafar and Anderson (in press) would later call an ethical-professional accountability orientation in a professional practice that had long been dominated by policy based in what the authors referred to as an economic-bureaucratic accountability mindset. At the same time, in conversation with district review participants I perceived that the framework provided a forum to tacitly understand that accountability practices in education could create "psychic prisons" or "emancipatory conditions", as Macpherson (1996) put it so vividly in visceral terms. The review process would seem to have demonstrated that deconstructing predisposed assumptions of accountability could 156 transform the concept. A more formal analysis of the dynamics surfaced through the development and implementation of the ABE accountability framework follows in Chapter Six. 157 Chapter Six Denouement (An Unraveling of the Plot) The philosophical basis for the EdD program and foundation for this research was reflexive practice. To better understand the possible reasons for the framework's current state of disuse this chapter addresses, in an introspective and theoretical manner, the meaning of the framework and the possible reasons that contributed to its outcome. I begin by outlining my role in the study, the expectations the ministry and the field might have held for me and that which I held for myself. In the context of contemporary events I then provide a theoretically grounded retrospective discussion of my experiences in designing the accountability framework. My Secondment: Leadership in Context Looking back on my work, I see the framework's development and evolution to be an attempt on my part to provide leadership in a context that was conflicted by a number of power dynamics. As a seconded Director of Instruction: Adult and Continuing Education, my role with regard to the Ministry of Education (MoE) was to be the Provincial Coordinator for Adult Education, whose primary responsibility was to liaise with the field and the ministry to ensure a more student centred and accountable delivery system was created for adult basic education in British Columbia. The expectation, as I perceived it, was to interweave my knowledge of the needs of both the field and the ministry to result in a holistic, well informed policy. As a secondee I was still a member of the British Columbia School Superintendents' Association (BCSSA) and the British Columbia School District Continuing Education Directors' Association (CEDA). At the end of my secondment, tentatively June 2000, I would return to my school 158 district. I knew the decisions I made or influenced as a secondee had to be defensible ethically and philosophically to ensure I could maintain my credibility to the ministry and the field. To succeed, I believed I had to generate agreement among all parties as to the value of the ministry's ABE initiatives. The power dynamics of the ministry were different from what I had been accustomed to in the school districts. As a school district senior director, the results of my decisions were relatively less complicated politically and much easier to design and implement. I often provided leadership in developing and executing visionary educational programs that afforded learners with cutting-edge programming. This was largely due to an organizational context where I exercised considerable power and autonomy with regard to the control of the resources I needed to implement my plans. As a secondee, my leadership role was difficult to define. My capacity to make policy decisions was more circumscribed. The ministry's organizational context afforded me little autonomy or control with regard to making policy, and was complicated by a number of variables that impacted my ability to provide leadership. There was first the ever-present party politics which complicated the ministry's administration and decision-making processes. The careers of many bureaucrats were dependent upon supporting the philosophical mandate of the party-elect, which could and did change like the wind, creating a persistent degree of instability with regard to the design and implementation of policy initiatives. There was secondly the hierarchical power structure of the policy decision-making process which comprised some individuals who were consumed by, and protective of, their domains. This resulted in what was often best described as a dysfunctional decision-making process, producing delays and problems with information 159 diffusion across departments. Third, there was a lack of communication between the field and the ministry which created a void with regard to implementing policy with practical application. This promoted a climate in both the ministry and the field that was somewhat conditioned to expect impotent policy initiatives which served no one and ultimately expended valuable resources.1 In many ways the leadership I could offer in the ministry was premised on providing policy advice within a very complex and paranoid organizational context. The Policymaking Process I approached the policymaking process understanding that it would be influenced by a number of variables that would impact its design and implementation. I understood focus, flexibility, and responsiveness to be characteristics of effective policy, and believed policymaking would be most potent if seen as legitimate. Following Gastil (1997), I thought legitimacy could be achieved by incorporating contemporary democratic approaches to policymaking to influence people in a manner consistent with principles of self-determination, inclusiveness, participation, and deliberation. Along with Wheatley (1994), 1 saw democratic leadership as being the most effective form of leadership in the long term as such organizations created "stakeholders" from "customers" and other "revenue sources" by willingly giving these people "voice" in decision-making, listening to different interpretations of problematic areas and desirable outcomes, maintaining focus rather than hands-on-control, and creating flexibility for responsiveness. These principles were inscribed into the ABE framework, giving stakeholders an opportunity to participate in establishing the meaning and parameters of accountability. This, I believed, would strengthen the integrity of school district delivery of ABE programs and reinforce their primary purpose to be the provision of high quality education to marginalized adult learners. 160 Through my experiences as a senior administrator and former teacher, as well as through conversations with colleagues and theoretical readings of the literature, I understood, as Grundy (1992) noted, that test-driven, standardized approaches to educational accountability which had predominantly been the preferred model for governments had too often resulted in the reification of learners and commodification of educational programs. Yet I knew, as Macpherson (1996) pointed out, that I would need to temper my enthusiasm for qualitative indicators of student success with more conventional thinking for the framework to succeed. While I saw some measure of accountability as achievable within conventional canons, I agreed with Thompson (2001) that the outcomes of these models tended to ignore the context of the educational environment and the more eclectic demonstrations of success in ABE programs and learners. As Downey (1988) noted, the policy process was not linear nor did it subscribe to a particular paradigm. It bore close affinity to Weiss's (1991) enlightenment model by providing a general perspective on problems and issues that were informing but not prescribing. The development of an ABE accountability policy had, as Rubenson (1994) suggested, strong possibility for giving rise to new models, rules and routines, curricula, methodology, and resources. This was particularly apparent in the school district review of the accountability framework conducted by ABE providers, where the participants put forward well thought through alternative pathways to graduation, innovative curricula, and nontraditional indicators of student success. I ascribe these results in part to Macpherson's (1996) observation that the concept of accountability is a social construct with a dual nature that makes it possible to work within society's value systems while simultaneously transforming them. 161 The complex nature of ABE program delivery and the diverse backgrounds of ABE learners meant that designing an accountability framework capable of capturing the multidimensional aspects of development required multiple understandings and approaches (Macpherson, 1996; Torrance, 1997). As this is time consuming and intellectually taxing, it has been the tendency, as Scheurich (1994) commented, for policy methodologies to not look for the complex, long-term solution, but for an easy quick fix. It was therefore exceptional for the Ministry of Education under the aegis of the NDP government to permit the design of an ABE accountability framework that could address diversity and the constancy of change over the long view. In contrast with Rubenson's (1994) observation that policy was most often reactive, here was a case of proactive development. Constructing the Framework The ministry's decision to design an ABE accountability framework appropriate to the needs of school district program providers launched the Provincial School District Review, which I conducted. As discussed in Chapter Four, the Provincial Review revealed there were few standards in the school district ABE sector. Some districts were awarding their own Dogwoods, some were passing ESL students off as ABE students, and others were representing students with grade 5 competencies as having passed grade 11 requirements. This created baseline problems with regard to the effectiveness of a ministry accountability process. In proposing the construction of an ABE accountability framework, the Provincial Review sought a means to standardize the reporting and program delivery issues that were conducive to standard practice across school districts while accommodating legitimate reasons for a district's departure from standard procedures. The review was concerned that the framework recognize and accommodate 162 the socioeconomic, demographic, geographic, and culturally diverse factors of each district which impacted upon their different rates of student progress. My involvement in the development of the MoE accountability framework was very much aligned with what Ben Jaafar and Anderson (in press) would later identify as an "ethical-professional" approach to understanding ABE accountability. This orientation was strongly supported by the then MoE manager of ABE and the Assistant Deputy Minister responsible for ABE. Both individuals had previously been with Ministry of Advanced Education, Training and Technology, and had a professional understanding of ABE attained from dealing with the college system. Both understood the philosophical conflict of the andragogy vs. pedagogy nature of ABE learners caught in a K-12 understanding of "economic-bureaucratic accountability" measures. The ABE framework was developed in collaboration with the ABE Accountability Committee and professional practitioners directly involved with adult learners in school district ABE programs. Through associations such as CEDA, discussion on student success was shaped by a number of filters that included ethical-moral, educational, and political considerations which tried to capture the voice of ABE students and professionals, and balance these with the MoE's pedagogical understanding of accountability in an andragogical learning environment. The framework's construction was greatly facilitated by the members of the ABE Accountability Committee being united in a common goal to create an equitable framework with emancipatory possibilities. The committee members, including two ministry officials with knowledge and experience of ABE, a director of continuing education representing CEDA, and myself, were more often than not on the same page with regard to how this goal might be accomplished. 163 The ABE Accountability Committee shared what Macpherson (1996) called a holistic and causally interdependent view of teaching, learning, and leadership in the formation of educative policy. Macpherson's consequentialist moral theory and its applications to education leaders and the policy proposals they develop and promote substantially influenced the process and content of the framework's design. Subconsciously, I was striving to meet his guidelines for effective educational accountability policy design. Thus the framework would seek to reflect processes and criteria that were educative and technically sound; provide opportunity for collegial judgement and expert input; and celebrate client interests, problem solving, and the growth of knowledge about learning, teaching, and leadership(p. 103). This understanding shared much in common with what Ben Jaafar and Anderson (in press) subsequently called the ethical-professional orientation toward accountability proposals. To encourage conditions for emancipatory development, the ABE accountability framework's design was premised on Macpherson's understanding that educational accountability was a social construct that could be deconstructed and reconstructed. The aim was to design a model that was "provisional" and "interperspectival" in nature. This was accomplished through an accountability proposal designed to be in continuous engagement with plural alternatives, bringing in ministry and school district perspectives and ascribing importance to traditional and nontraditional indicators of success. The longer term intention was to include in the accountability reports to government the perspectives of students, the business sector, First Nations communities, religious groups, and other stakeholders. The goal was to enable ABE providers to legitimize program success and encourage adult students to re-engage in learning activities that nurtured their desire for further learning. 164 As a social constructionist Macpherson understood accountability policies were most commonly a means of maintaining the status quo and keeping the power of privileged groups intact. Such polices were oriented toward making incremental improvements to a system's operations but not toward questioning the value of the system itself. He advocated moving past this form of "systems thinking" where the assumption was that schools comprised an objective reality that could "be improved by refining production functions, adjusting management mechanisms, and giving epistemic privileges to empirical data" (p. 103). He suggested we might instead interrogate the received wisdom we continuously reconstructed through unconscious acquiescence and passivity, and reflect on whose interests were being served, whose were being denied, and why. Macpherson advocated understanding that "schools and school systems exist as multiple and complex networks... that comprise socially constructed beliefs and feelings, a moral economy of norms and values, and empirical knowledge" (p. 103). Networks of power relations and value systems have over time created a social structure that serves the interests of some while undermining that of others. He pointed to standardized testing as a central feature of "systems" approaches to education. He noted that House (1972, cited in Macpherson, 1996, p. 82) had argued that standardized testing was a means to maximize the interests of the powerful groups of society which neither met the needs of clients who were disenfranchised nor encouraged the development of responsively accountable schools. In the United States standardized testing has also been linked to funding, and administrators and schools have often had their funding envelopes tied to their students' results on standardized testing. This retributive form of accountability in educational bureaucracies has alienated teachers and administrators, leading to the creation of some very divisive school environments. 165 While Macpherson (1996) argued that educative accountability should be considered a complex social construct, Ben Jaafar and Anderson (in press) observed that accountability was a concept that was seldom defined in policy documents. This "definitional vagueness", the authors contended, provided an opportunity for "alternative accountability approaches to co-exist in relation to different dimensions of the education system" (p. 5). Following Spencer (2004), Ben Jaafar and Anderson proposed that accountability be conceived as having two orientations, economic-bureaucratic accountability (EBA) and ethical-professional accountability (EPA), with the understanding that accountability need not necessarily conform to either orientation but could fit into both. While EBA reflected a results-based model to maximize benefits with limited resources, EPA was a process-based model founded in public democratic discourse and reflected through mutual accountability. Ben Jaafar and Anderson argued that EBA and EPA were not polarized extremes but co-existed in a fluid and dynamic state of differentiation and interaction that supported the notion of education as complex, multidimensional, and contradictory (p. 8). Although the ABE accountability framework was created a few years prior to the EBA-EPA conceptualization of educational accountability, its intent was to bring together in a single accountability mechanism what Ben Jaafar and Anderson later referred to as the EBA and EPA orientations. That is, the approach the MoE Accountability Committee devised was intended to "coexist in relation to different dimensions of the education system", as Ben Jaafar and Anderson put it (p. 5). By opening the EBA system to other orientations the MoE Accountability Committee hoped that a more equitable and mutually respectful paradigm might evolve. While I have reservations with regard to the EBA-EPA model as enunciated by Ben Jaafar and Anderson, it provides a useful heuristic for understanding the kind of framework the MoE ABE 166 Accountability Committee constructed. However, the experience has taught us that the EBA and EPA orientations do not co-exist as Ben Jaafar and Anderson might imagine. Although Ben Jaafar and Anderson argued that EBA and EPA should not be considered polarized extremes, it must be noted that only one model is capable of disenfranchising and marginalizing students. Standardized testing, an essential pillar of support for an EBA worldview, is extremely polarizing when compared to anything in the EPA repertoire. The authors note that standardized testing aligns itself with EBA as it is oriented only to outcomes that "can be conveniently measured, and diverts attention from other purposes and goals of education such as good citizenship, social skills, technology competency, and preparation for employment" (p. 23). One might add that these "other purposes and goals of education" also include essential life skills that may make a difference between immiseration and a modicum of dignity for many learners enrolled in adult basic education programs. The impact of EBA can be even more polarizing when it is linked to funding. Currently, ABE students are funded on the basis of course completion rates, unlike school-age students (under 19 years of age) in the regular K-12 system who are funded as FTEs if they were in school effective September 30th, regardless of how many courses they had completed. This funding inequity between adults and school-age students in the same system represents a "systems within systems approach" to accountability where the ABE system is much less equal than the K-12 system. The rigidity inherent to the EBA model effectively marginalizes ABE students and their programs. EBA is able to do this because it is the predominant paradigm. It holds all the aces and is the one in control. 167 Ben Jaafar and Anderson's choice to analyze Canadian accountability practices through an EBA and EPA lens does not adequately capture the diversity within educational systems and the need for a socially constructed concept of accountability that reflects programs such as ABE. Their review of educative accountability through an EBA/EPA lens is enlightening, but they overstate the case when arguing that even though educative accountability practices in Canada are EBA-based "there are few, if any, direct consequences associated with the level of performance reported either for individual teachers, schools or districts" (p. 33). There can indeed be direct consequences of EBA. As mentioned in Chapter Four, in 1998, when the controversial Educom report was released, the MoE chose to use the survey's unreliable course completion rates as the basis for clawing back ABE funding. The claw-back was an EBA tactic; and although it was an anomaly at the time to use course completion rates as the basis for funding, it has now become the norm. At the time of the survey, district success rates for ABE were measured through course completion rates, but funded on the basis of the number of courses students were registered in, providing the school district could demonstrate attendance and progress. This provided school districts with the necessary funding to support the learning needs of marginalized learners with low literacy levels who required time to acquire literacy skills. The Liberal regime marked a departure from past practice: districts currently receive ABE funding on the sole basis of course completions. This is in line with the core tenet of the EBA systems approach, i.e., economic efficiency, or saving time and money. In adult basic education this means that students who attend regularly and progress to the best of their ability often do not receive full funding. ABE is commonly a slow, incremental process, frequently characterized by financial and personal setbacks on the part of the adult learners. They frequently take a longer time to achieve success. The OECD noted that even under the best of conditions, the transition from the lowest levels of 168 literacy toward participating somewhat effectively in the "knowledge society" required at least 12 months of full-time study (OECD, 2002, cited in Rubenson and Walker, 2006). ABE students in British Columbia have not been funded at anywhere near equitable conditions, few can afford the luxury of full-time study, and current provincial policy for school district ABE programs adds a further impediment by tying funding to quick turn-around values measured in course completion rates. While this has obvious implications for adult learners it also has direct consequences for teachers, schools, and students. The teachers, students, and administrators are forced to subscribe to a funding formula that results in continued underfunding and misused resource allocation for students who were initially failed by EBA systems approaches. This erodes morale, creates dissonance in the field, and has the potential to encourage a system of admissions testing whereby only those ABE students who can demonstrate higher levels of academic performance are enrolled into the programs, thereby marginalizing students with the lowest levels of education who are the most in need. The possibility for this type of marginalization was noted by Rubenson and Walker (2006) in their examination of how the neoliberal political-economic paradigm has transformed the meaning of adult learning in Canada. They concluded that while the Chretien government's "Canadian Way" made frequent reference to inclusion and equality of opportunity, participation data indicated that the most marginalized groups of learners such as Aboriginals, the working poor, and those with low literacy levels were the least likely to participate in adult education. Engaging in adult education for those with low levels of education was further complicated by the time commitments required of these individuals to achieve higher literacy skills. Obviously, a funding formula based on the average cost of course completions will not provide ABE programs with the resources they need to accommodate these learners. 169 The impact of EBA for ABE extends from one level of bureaucracy to another, in some cases from the MoE to the school boards. It was reported to me through public documentation that one school district with approximately 10,000 students and 28 schools had released its operating budget for the 2006/07 school year to gain public support for the closure of three schools. The information contained in the operating budget stated each school's cost per student to operate the school. The range in per student cost of operation amongst the 28 schools was $4,800 to $28,000, with the ABE school at $4,800. Through discussion with the program administrator I found that even though the ABE program was the most cost efficient in the district, it was expected to generate a minimum of 20% profit over and above its operating costs. This resulted in the ABE program being underfunded with regard to resource allocation. Furthermore, the district had the highest provincial pupil-teacher ratio, 54-1, as compared to a provincial average for school district ABE of approximately 40-1. The ABE framework through the program assessment would have surfaced these inequities and as such would have the potential to ameliorate the indifference of various levels of bureaucracy with regard to the further marginalization of those already marginalized. While Ben Jaafar and Anderson (2006) suggested that EBA and EPA were not really polarized orientations I would argue that in British Columbia's school districts under the current Liberal government we have an EBA model that is very polarized from EPA thinking. The idea of both models co-existing was an ideal of the ABE framework that briefly became a reality during the school district review. But the existing government's shelving of the framework made it apparent that the main reason dichotomizing alternatives to educational accountability may not accurately portray the practice of educational accountability (as Ben Jaafar and Anderson enunciated it) is because one orientation has the power to discount the other. The notion that educative 170 accountability practice is a co-existence of EBA and EPA orientations does not seem to take political realities into consideration. When considering the consequences of the EBA model, Ben Jaafar and Anderson were undoubtedly focusing on educational accountability in general, not ABE accountability in particular. While the authors discussed the consequences for students who failed, this was only in relation to a lack of consequences for teachers who failed them (p. 34). Ben Jaafar and Anderson attributed student failure to a scaffolding effect whereby the EBA approach was actually supported by a lack of "collective responsibility" in EPA "to genuinely assure professional competence in practice not just on paper" (p. 34). To suggest that educational professionals lack collective responsibility is provocative but lacks authentication. Based on my experience in the development of the ABE accountability framework I would argue that the unwillingness to challenge EBA is more an affect of governmentality and professionalization. When the participants in the school district review did not believe they had a voice in educative accountability, little was accomplished except for the reiteration of pat phrases the participants anticipated a government in power might be pleased to hear. But when the framework afforded the participants a forum to engage in an emancipatory process the result was a well articulated EPA framework, emancipatory in its intent. Within British Columbia's K-12 system today there is little opportunity to voice critique of a centralized autocratic systems approach to educative accountability. The issue is divisive with regard to administrators and teachers. Administrators are directed to subscribe to government policy and there is no effective discussion. School boards can be removed for failure to comply with MoE policy. The current Liberal administration has introduced so many accountability 171 initiatives that by virtue of administrative overload school districts are always trying to keep up with accountability paperwork that has nothing to do with improving performance, and more to do with keeping the system so overwhelmed it is unable to mount a collective response. An example is the Parent Advisory Councils (PACs) that were introduced by the Liberals. The PACs exist to work with school staff toward school improvement, arriving at yearly goals and objectives such as improvements in literacy within a particular grade level. The MoE first introduced this policy oblivious of the existence of ABE programs but later insisted that ABE administrators and teachers must also have PACs. By definition, and as described in the literature, ABE students are adults, often parents themselves, not children in the custody of parents. Furthermore, ABE students are highly transitional. Their attendance is problematic. As the school culture and knowledge base is continually changing, it is impossible to predict from the current student population what the literacy or numeracy ability will be of the students attending the program next year, let alone plan annual goals and objectives for improvement. Put simply, the ABE program is not an elementary school. The notion of a PAC for ABE students is absurd and nothing more than an administrative travesty of EBA orientation. It mocks ABE practice and thereby marginalizes it. Oversight is common throughout much of the research on educative accountability when it comes to school leavers. ABE is not at the forefront of anyone's agenda. It has no priority in government or anywhere. That is the definition of a marginalized system. There is no voice among the powerful and the well-educated for ABE. It can only become apparent through a socially constructed notion of educative accountability that is all-inclusive of the learners impacted by a system. The EPA/EBA lens, as Ben Jaafar and Anderson have explicated it, does not seem sufficiently subtle, perceptive, or reflexive to capture and unpack a system within a 172 system such as ABE. What is needed is not only an "interperspectival" lens, where the perspectives of others interpenetrate the system, but a "provisional" one that is more organic than mechanical, adaptable to shifts in accountability concepts, populations of learners, and funding restrictions. In my view the ABE framework came closer to meeting this need. The School District Review The impending election in the final year of my secondment created a somewhat unstable work environment. Discussion with colleagues who were established bureaucrats alluded to the possibility that the ABE accountability framework could be shelved as experience predicted that changes in government often resulted in changing political ideologies. Projects such as the ABE accountability framework were thought to reflect different ideologies from the incoming administration and could subsequently be laid to rest. The possibility that the ABE accountability framework would not be followed through created both a sense of pathos and urgency. I believed that the framework would provide the Ministry of Education with a legitimate and utilitarian understanding of ABE accountability, even though I knew it could be shelved. I believed that if I could expedite the development phase to the implementation phase the efficacy of the framework would become self-evident. I believed that implementation through a school district pilot would demonstrate to the field and CEDA that the framework was an educationally sound accountability tool for school district ABE providers and that their support would influence a different government to acknowledge, however tacitly, a further review of the framework. 173 The school district review of the student assessment aspect of the ABE framework became a catalyst for reflexive practice as the teachers struggled with years of marginalization with regard to their input into an inflexible EBA practice. Through group dialogue the participants began to transcend the influence of governmentality and professionalization by disassembling the emancipatory and punitive aspects of accountability and rebuilding the concept in a way reflective of their professional practices and beliefs. Through communicative action initiated by the ABE framework the participants went from feeling alienated by ABE accountability practice to becoming advocates for a decentralized curriculum, the recognition of student diversity, and indicators of student success that included voice and power. Through the school district review process the ABE framework demonstrated a strong EBA orientation with regard to its program assessment component. Reviewing the student assessment aspect resulted in success indicators that were EPA oriented in that they did not reify learners but rather valued individual difference and understood that different indicators of success were required. These were discussed by the review team participants as emancipatory indicators of student success that, for many ABE learners, were foundational to future success within their educational endeavors. The school district review team not only isolated a number of nonacademic indicators of success but were also able to catalogue and describe them through anecdotal comment that demonstrated their relevance with regard to an ABE accountability model. Through discussion with the participants it appeared evident to me that as the teachers argued for the emancipatory voice of their students they were also emancipating themselves from the punitive nature of an inflexible EBA model by engaging in a reflexive process with regard to their professional practices and beliefs. The process of answering the framework's questions on accountability would either force educators and their assessors to accept entrenched values or enable them to review, revise, develop, and 174 strengthen the values they brought to the assessment enterprise. In this sense, the accountability framework was an expression of the potential Macpherson foresaw to build psychic prisons or encourage the conditions for emancipatory development (p. 92). Political Realities However rushed and substandard the review process was, it was the only means I had available to test the efficacy of the framework. The review process yielded results beyond my highest expectations. But the framework never took root. What I failed to take into consideration was the timing of the framework's implementation coinciding with the onset of a new political reality. In the dying days of the NDP government a change in administration was discussed throughout the bureaucracy as a reality. The pervasive climate of political uncertainty extended throughout the bureaucracy, resulting in there being little political decision making with regard to new policy initiatives. Although I received continued support through the completion of my secondment and the final ization of the ABE accountability framework, I was a practitioner not a bureaucrat and politically naive with regard to the timeframe of policy implementation in a politically uncertain environment. Although I finalized the ABE accountability framework, it was too late to do a formal pilot. If a formal pilot of five districts had been done the framework would have been better connected with the field and more politically charged. CEDA would have been given some ammunition and in turn could have swung its weight behind it. The fact that I was able to engage a school district to formally review the framework over a five-week period did nothing to support the efficacy of the framework within the new government. The end of my secondment coincided with the demise of the NDP administration. With that, any voice I might have had with the ministry was effectively muted. 175 When the NDP government fell to the Liberals in 2001 the new regime's platform favoured an even more draconian economic-bureaucratic orientation toward accountability than the EBA model that had been in place with the NDP government. This reflected a shift in political orientation from the NDP's more social democratic understanding of government as holding a mandate to work toward the reduction of poverty (Jary and Jary, 1995). This worldview was reflected in my secondment to create a more coordinated system of adult basic education and high school graduate certification across the college and school district delivery systems. At base was an agenda to improve the learner's access to equitable, high quality educational services as a means to enable the individual to make further life style improvements. The same could be said of government's decision to extend my secondment for an additional two years to design an accountability framework for ABE offered through school districts. The victorious Liberal administration's understanding of government as a strategy of social deregulation and marketization encouraged the opening up of new arenas of social, cultural, and intellectual life to privatization and capital accumulation, as Lipman (2006) noted in general of neoliberal regimes. The framework was shelved before it could be either an emancipatory or disciplinary device. The Liberal administration chose a more easily understood, more quickly implemented means of assessing accountability even though the information provided by test-based assessments and course completion rates would be inadequate for improving educational practice. The current accountability model of British Columbia's school district ABE programs is a systems approach whereby funding is tied to course completion rates. This approach is incapable of measuring or improving upon the qualitative aspects of student success that are essential for academic progress, and has the potential to further marginalize those adult learners who are most in need of fundamental literacy skills. 176 The Current Situation At present, publicly offered ABE programs in the province of British Columbia hold an uncertain status. While both the school district and college sectors continue to develop the professionalism of their ABE programs, both have experienced continuous funding cuts from the neoliberal policies of BC's Liberal government. ABE remains the "poor cousin" in the educational bureaucracies of both sectors, which share a common disinterest in undereducated adults and are habituated toward continually looking for opportunities to siphon ABE funding into their preferred political agendas, being school-age students in the school district K-12 system and postsecondary students in the college system. In 2006 the Federation of Post-Secondary Educators (FPSE) submitted the report "Opening Doors and Building Confidence" which outlined its proposals to strengthen ABE in British Columbia (FPSE, 2006). While noting a number of areas that had negatively impacted the federation's ability to offer ABE, the report resurfaced the turf war between college and school delivery of ABE by claiming that school districts were not a fitting milieu for ABE as "adult learners should learn in an adult environment and the public post-secondary system provides the appropriate learning environment for these students" (FPSE, 2006, p. 4). This demonstrates that colleges continue to suffer from a malaise of misunderstanding. School districts do indeed provide adult learning environments and adult learners have in fact always had access to separate and distinct adult facilities that in some instances were a fair distance from K-12 facilities. Although the FPSE report was intended to influence the opinion of both education ministries, the content of the report's argument did little more than divide the advocates of ABE learners. 177 It is within government's interest to maintain a certain level of dissonance between the systems because the more fractured this group of professionals are the more room government has to manoeuvre. Both ministries would save a great deal of funding by turning ABE over to a private education vendor which might not pay the wages under the collected bargaining agreements of the Teachers' Federation or the various college instructors' associations. Given that both the school district and college sectors have come under increased attack over the past two years the need for a unified front is stronger now than ever. So much more could be accomplished if the two systems worked together, as was evidenced in the Transitions Project. The efforts that went into that attempt to build a coordinated system revealed the college system had far more understanding of the need for different assumptions for ABE students than did the MoE. Although the process of developing a common accountability framework for both ministries was not practical due to inequities such as funding, the goals and objectives that arose from the committee reflected Ben Jaafar and Anderson's (in press) ethical-professional understanding for ABE accountability (see Table 4.1). This was epitomized by the ABE educators of both sectors acknowledging that the failure of educational structures was the primary cause of student failure. This contradicted Ben Jaafar and Anderson's general observation that "[t]eachers' professional accountability remains situated firmly in... [a] paradigm... [that applies]... high stakes consequences for students who fail, with little or no consequences for the teachers who fail them" (p. 34). In advocating a model that ascribed responsibility for student failure to themselves and other system components, the teachers were acknowledging their own culpability and asking that it be inscribed in the framework with no likely gains for themselves in terms of professional status or security. 178 In spite of these moves toward professional accountability, the MoE K-12 accountability system still holds the students accountable for their own success. The lens for determining success continues to be limited to quantified results on standardized tests and course completion rates. Schools are also rated on their successes within these narrow parameters. To shift the accountability mindset of the MoE from an economic-bureaucratic understanding of accountability has proven to be unattainable even though the work of the ABE Transitions Project clearly demonstrated the need for different ways of understanding ABE accountability. The notion that there are other dimensions signifying learner and program success surfaced in the Perspectives on Work, Education and Responsibility Symposium (Huget, 2002). The symposium brought together 28 representatives from British Columbia universities, colleges, and school districts to listen to business, labour, and community groups discuss employability skills. Along with numeracy and literacy skills, the presenters identified a number of basic, soft, and leadership skills and attributes, including responsibility, ability to multitask, critical thinking, self-motivation, punctuality, self-confidence, flexibility, positive attitude, creative thinking, independence, ability to motivate others, resourcefulness, and learning how to learn, to name a few. It was determined that educational programs and accountability must ensure that the citizenry were enabled with these basic skills and attributes to meet "the realities of the economy and the world of work" (C. Kelly, BC School Superintendents Association, cited in British Columbia, Hansard Debates, 2001). Overcoming barriers to employability and enhancing the lives of British Columbians is a key step in this direction. It might also appear to be held as a central tenet of the current government. 179 On September 14, 2005, the Minister of Employment and Income Assistance, Claude Richmond, stated that In addition to providing assistance to those most in need, we continue to help people who are able to work to achieve sustainable employment. We will help British Columbians with barriers to employment to overcome those barriers and find good jobs. By helping our clients achieve independence, we are assisting them to build brighter futures for themselves and their families, (cited in British Columbia, Ministry of Employment and Income Assistance, 2005) However, recent government initiatives with regard to adult basic education do not indicate the current administration is particularly concerned with lowering the barriers these learners have to educational participation, increasing their access to further learning, or helping them to lead more fulfilling lives. Quite the opposite. In 2006 the provincial government commissioned a private consultant, Gail Thomas, to conduct a review of school district and college A B E programs. In August that year she submitted a report which indicated that school district A B E programs were more successful with regard to course completion and graduation rates than had been previously thought (Thomas, 2006). In November 2006 there was informal discussion between the Deputy Minister and CEDA with regard to the findings of the Thomas report. These discussions included the strong possibility that there would be a funding increase for school district ABE programming. This meeting was held in addition to several CEDA meetings with other ministry officials regarding system collaboration, accountability measures, and funding for adults - which included funding of adult ESL courses and a two-year incremental phased-in increase in adult funding to match the school-age and distance learning funding levels. Al l looked positive in January and February 2007, when CEDA participated in four regional meetings on system collaboration and co-presented with its counterparts from the Deans and Directors Association on K-9/Developmental and Trades 180 Training Programs at the meeting o f School District Superintendents and College Presidents on February 21. However , on M a r c h 2, 2007, at C E D A ' s meeting with M o E officials with regard to adult student funding, it was made clear that all moves in this direction were coming to a halt: there would be no increase in adult student funding after a l l . This was done without consultation with C E D A and without prior warning. The reversal in the funding lift that C E D A had been anticipating w i l l negatively impact the future o f school district A B E delivery. There is a great deal o f speculation about the meaning o f these actions. It may be that government w i l l impose new standards on school district delivery o f A B E that favour the college system. If this is the case it may be possibly due to the college presidents currently wielding more power with government than do the senior managers o f school districts. The use o f the P A C in school district A B E program delivery would then be a useful device for college A B E program providers to bui ld argument that the M o E and the school districts know little about A B E learners, and that the A B E programs would be more competently handled by the college system and the Min i s t ry o f Advanced Education. However, there is a more probable scenario. A long-term strategy aligned with neoliberal thinking would be to cut funding for school district A B E delivery and continue to further obfuscate funding for college A B E programs through administrative policies that steer college administration toward postsecondary education programming. The postsecondary public sector would be rendered impotent wi th regard to accessing A B E funding. This would be consistent with neoliberal philosophy as government could then argue that turning A B E over to a private provider was its only alternative. It is also interesting that C E D A has chosen at this time to acquiesce to government's decision to starve A B E in the school districts. Perhaps due to the 181 disingenuous nature of the current government and its loyal ascription to neoliberal policy, the organization appears somewhat traumatized. Summary In the context of contemporary events this chapter provided a theoretically grounded retrospective analysis of my thoughts and experiences as a seconded "social engineer" attempting to construct a more equitable, possibly even emancipatory way of conceiving student and program success in adult basic education accountability proposals. Whereas Macpherson's (1996) consequential ist moral theory was of inspirational importance to the construction of the framework, Ben Jaafar and Anderson's (in press) EBA-EPA construct provided a means to look back on my work. So, was the framework successful? Arguably, it was successful at eliciting discussion around accountability and unpacking how teachers were being regulated by professionalization and governmentality. It urged them to begin thinking differently, and in doing so, demonstrated great potential for implementing institutional change in educational and administrative methods with regard to adult basic education. Yet the teachers never acted on the new curricula designs, graduation pathways, and measures of student success they devised. They needed leadership from both the field and the ministry to express their creativity and overturn the forces of governmentality. The frustrating thing for the teachers was that they couldn't go further with the exercise - the clearance for that never came. Once the Liberals were elected the opportunity for an EPA oriented ABE accountability framework was all but eliminated and government accountability in general was firmly grounded in a rigid EBA model. 182 The one-size-fits-all EBA approach to educational accountability has done little to encourage student retention; the dropout rate is still twenty-five to thirty percent in the public sector (British Columbia, MoE, 2001). Perhaps even more remarkable is that when student leavers return as adults they are then introduced into an ABE system where educational success is determined through an EBA approach that is even more narrow minded and cash starved than the system that failed them to begin with. It is within this group of learners that a broadly "interperspectival", morally responsible social construction of accountability is of particular significance if governments are sincere in assisting adults to gain an education. Such an inclusive perspective might encourage governments to think about education in terms of its social consequences and thereby go beyond simplistic empiricist arguments based on test scores and course completion rates, even though these measures appear more tangible than personal development. British Columbia's school district ABE accountability and funding practices have created a dynamic to marginalize ABE learners through an overly pedantic EBA model. In this province's school districts under the current Liberal government we have an EBA model that is very polarized from EPA thinking and unwilling to accommodate anything other than its own perspective because there is little pressure on it to do otherwise. Ben Jaafar and Anderson's discussion of both models co-existing became a reality briefly during the school district review of the ABE framework. However, given the rejection of the framework by the existing government it would appear that government rhetoric around BC becoming the most literate province in Canada is little more than hyperbole. It is possible that existing neoliberal policy will be expressed through increasingly aggressive and centralized EBA models of educational accountability that may result in continued marginalization of already marginalized learners. 183 While Ben Jaafar and Anderson argued that dichotomizing alternatives to educational accountability do not accurately portray the practice of educational accountability, I would submit that EBA and the resulting systems approaches to educative accountability have dire consequences for the students impacted by them. 184 Notes While this situation remains unchanged, some steps for its improvement were taken during my secondment with the creation of an education officer's position (on whose hiring committee I served) who continues to act as liaison between the ministry and the field. This individual's ability to enlighten the political administration of the day is necessarily limited. 185 Chapter Seven Central Findings and Implications This chapter summarizes the study in three stages. It first provides an overview of the study, linking the central tenets and understandings made in the previous chapters. In light of the implications highlighted in the study, it then makes recommendations for the practice of accountability within school district adult basic education (ABE) programs. Finally, it proposes possible research directions that could expand upon the understandings gained from this study. Overview The purpose of this research was to develop a deeper understanding of accountability in adult basic education. It focused on the policies applicable to the ABE programs offered through public school districts falling under the aegis of the Ministry of Education in British Columbia, Canada. Depending on the prior educational level of ABE students, their journey through formal schooling can be lengthy, commencing with fundamental literacy courses, or it can be relatively brief, with students beginning in grade 11 or 12 courses. Regardless of their entry level, their sojourn has the potential to change their lives forever. Consequently, understanding how to help students maximize their experience in ABE programs and possibly develop a thirst for further learning is of the utmost importance. Beginning in the mid 1990s, the Ministries of Education and Advanced Education sought insight into ways that might best support the educational endeavours of ABE students. By 1998, these efforts resulted in bringing together representatives from the two ministries and from school district and college ABE programs in an attempt to build a more cooperative and well-articulated 186 system of ABE delivery in the province. During my three-year (August 1998-September 2001) tenure with the Ministry of Education as a seconded school district director responsible for ABE programs, four policy goals were focused on to increase collaboration between school district and college ABE providers: developing a common credential (achieved in 1999 as part of my responsibilities); improving course articulation between the K-12 and postsecondary ABE programs; implementing a regional planning model that would facilitate cooperation between college and school district ABE providers at a local level; and creating a common accountability framework applicable to both systems. The goal of building a common accountability framework could not be accomplished because of profound systemic differences. This led the Ministry of Education to pursue the design of an accountability framework specifically applicable to the province's school districts. The design of this framework evolved over a two-year period and was overseen by a provincial subcommittee made up of ministry and field representatives, including myself. This dissertation examined the theory, design, and evaluative practices that went into developing this accountability framework. The approach was a retrospective case study that focused on the rationale, evolution, and assessment of an ABE accountability framework. It was "interperspectival" and inclusionary in orientation as it solicited insights from a number of ministry and field representatives as well as from conventional and unconventional theoretical approaches to accountability. A review of the literature related to educational accountability and adult basic education initiated the study to determine what was already understood about the problem. The literature review was also pivotal in informing the design and development of the accountability framework produced as a result of my time at the ministry. 187 Numerous studies were reviewed that dealt with the difficulties of matching educational accountability measures with the challenges many adult learners brought to school district ABE programs. These studies examined the nature of educational accountability, the nonacademic gains valued by students and understood by researchers as foundational to present and future success within educational endeavors, and the various educational accountability alternative models that might better represent student success within ABE programs. The review was guided by one broad question; what influenced the development and fate of the ABE framework? The question was addressed by examining the effect the political economy had on the process, the product, and the framework's development and the socio-political structures that influence accountability within publicly funded education programs. Political economy was defined as the social relations or power dynamics that exist between political, economic, and educational interests participating in decisions affecting the framework's development, evaluation, and end result. Secondary themes identified through the literature included the significance of nonacademic values intrinsic to adult learning, the psycho-social emancipatory and transformative outcomes demonstrated by students enrolled in ABE, and the socioeconomic implications of educational accountability for ABE students. These themes provided a means of structuring the information gathered for this study as well as a basis for conceiving, designing, building, and evaluating an accountability mechanism that was intended to provide the best support for ABE student success. Previous research had indicated that traditional assessment mechanisms were often "decontextualized", or rather, contextualized for a standard and situation in which many could 188 not reasonably be successful. In focusing on controlling performances rather than understanding them and enabling transformation, traditional performance measures often failed to appreciate that student success within ABE programs was predicated on nonacademic gains such as self-esteem, emotional literacy, and self-knowledge (Quigley, 1990; Upex, 1999). Conventional approaches to introduce educational accountability mechanisms to adult basic education programs were often predisposed toward measuring success through standards that failed to recognize nonacademic achievements (Thompson, 2001). As educators and educational theorists tend to understand academic gains to be a secondary outcome of acquiring more fundamental skills and personal attributes such as self-confidence, resiliency, and perseverance, it appears that traditional models of accountability and assessment might be unwittingly setting up adult basic learners for a lifetime of failure. The problem, however, is not with educational accountability per se, but with how it has been defined, structured, and executed. A great deal of research has supported the need for accountability in educational programs such as ABE (Boone, 1985; Macpherson, 1996; Merrifield, 1998). As revealed in Chapter Three, Thompson (2001), Glass (1972), Macpherson (1996), Knowles and Knowles (2001), Eisner (2000), Pipho (1989), Torrance (1997), House (1972), Levin and Kelly (1994), Pacquette (1998), and Grundy (1992) have all understood educational accountability in its conventional form to be a highly enigmatic and potentially reifying concept that has been too often limited to standardized high-stakes test-based assessment. These authors brought more holistic approaches to educational accountability which valued and empowered the learner through success indicators that were not limited to measurable systems-driven academic outcomes such as standardized tests and course completion rates. 189 This literature, which emphasized understanding the complex nature of educational accountability, was foundational to building argument for a more student-centred approach to dealing with the ABE accountability issues that surfaced through the provincial review. However, within the political nature of publicly funded educational programs, test-driven quantitative academic indicators of success were a political reality. An effective ABE accountability framework would be prudent to incorporate both academic and nonacademic indicators of student success. The evolution of the ABE accountability framework was an iterative, ongoing process of analysis, design, testing, and refining, all of which occurred simultaneously. Through the provincial review of school district ABE programs, the field enriched the process with its voice and expertise. The Ministry of Education, as the sponsor and funder of the accountability review process, responded to insights gleaned in the provincial review yet maintained a firm hand in its expectations with regard to the political challenges of ensuring that the framework would be bureaucratically credible. The catalyst for this research surfaced through the 1998 Educom survey of ABE course completion rates (Reed and Associates, 1998). The survey's results promoted an ABE provincial review. The review revealed incommensurable levels of diversity among the regions of British Columbia as well as a number of ABE accountability issues. It became clear that to apply a "one size fits all" model of accountability would merely gloss over regional and geographic differences, socioeconomic differences, and the demographic differences of the student body with regard to educational and cultural backgrounds and expectations. 190 The MoE's ABE accountability committee provided a formal arena for field and ministry representatives to review the design and analysis process of the ABE framework. The consultative measures taken during the design process best ensured acceptance by the field and the ministry. These measures also had the potential to infuse ABE accountability with a long-term policy solution. Such a solution had the possibility to be student centred, and with this focus thereby resolve many of the issues captured in the provincial review at the same time as incorporate standards entrenched in the political sphere of publicly funded educational programs. Analysis of the problem was guided by the ministry's ABE accountability committee. Discussions at the committee level and with pertinent field and ministry representatives resulted in a continual process of design, analysis, and refinement. These discussions provided a discerning and insightful forum for understanding the dynamics involved in educational accountability within the context of provincial ABE programs. Issues that the research suggested were intrinsic to the design of an effective accountability framework such as the social context of ABE environments and recognition of nonacademic outcomes were raised in these forums. Part of the analytical process was a school district review of the ABE framework which included a number of district volunteers. This process was abbreviated due to a change in government. However, the MoE Accountability Committee members felt that as so much work had gone into the analysis and design of the ABE accountability framework, even an abbreviated pilot might provide insights that would lead to deeper understandings of educational accountability within the context of school district ABE programs. The ministry's ABE Accountability Committee and the school district review embraced a dialogue process that promoted understanding the complexity of determining multiple outcomes for ABE program success. 191 Analysis of the school district pilot process indicated that even in its abbreviated form the ABE accountability framework effectively addressed a number of issues surfaced in the Provincial Review. The many technical aspects of the program such as teacher certification, attendance, and tracking of student academic progress were addressed through the framework's program assessment component. The framework seemed to be an effective tool at enabling districts to articulate regional anomalies such as public transportation that might be different from other districts and could impact ministry expectations with regard to attendance. Transparency with regard to budgeting and allocation of ABE funding was vetted through this accountability mechanism as were numerous other quality assurance issues articulated in the framework's design. As 1 understand it, the enigmatic nature of educational accountability that surfaced in the literature re-emerged in the school district review process without solicitation from, or formal review of the literature by, the review participants. The reification of students through standardized testing was identified as a problem within the social context of ABE. The failure to recognize other student gains such as self-confidence was discussed as limiting the effectiveness of existing ABE accountability practice. As the committee members became more involved in the analysis of the ABE accountability framework, they developed a collective notion about ways to enrich existing ministry accountability policy with regard to indicators of student success that were more representative of this particular ABE environment and its students. The school district review process became a catalyst to encourage debate and understanding about the purpose of education and the implications of accountability for educational practice within the context of an ABE environment. Although the literature had in theory prepared me for 192 it, it was still surprising and gratifying to discover that the school district review participants had in a brief five-week timeframe engaged in lengthy conversations regarding the purpose of adult basic education and the influence of sociopolitical structures in arriving at and legitimizing nonacademic success indicators. They concluded that nonacademic indicators of success were the most important for many ABE students, and they identified a number of nonacademic success indicators that they felt should be included and recognized as indicators of success in school district ABE accountability. The district review members' understanding of educational accountability in some senses seemed to be more transformative and emancipatory as they unpacked deeply ingrained notions of accountability. In many ways their discussions and actions appeared to imply a renewed, or perhaps emergent, sense of awareness of deep-seated sociopolitical doctrines and structures such as governmentality and professionalization that influenced the social order (Scheurich, 1994; Delandshere, 2001). This seemed particularly evident in the committee's summation of ABE student success through nonacademic learning outcomes (Table 5.1) and their articulation of alternative graduation pathways. This overview focused on the factors that impacted educational accountability within ABE environments and the commonalities of student success that appeared to exist across divergent ABE settings. Insights gained from the study of these factors and commonalities have been documented and analyzed. Consideration now turns to the possible implications these insights have for the field of adult basic education. 193 Implications for Educational Policy and Practice This study's findings pointed to a number of social, political, economic, and contextual issues that have implications for the practice of accountability within adult basic education. Politically, a realistic accountability framework would provide government with an assessment process to ensure that ABE providers were effectively utilizing public funding for the development and implementation of successful ABE programs for adult learners. From a practical perspective, it would also provide the field with an advocacy mechanism that legitimizes the success of its ABE programs. Education and lifelong learning are becoming increasingly more influential in determining long-term personal and economic benefits for the individual and society (Faris, 1992). An ABE accountability process that fails to address the educational abilities and needs of those who have difficulty demonstrating learning gains through traditional K-12 assessment structures will continue to marginalize those with the lowest levels of education (Wrigley, 1998). This is an important political and social consideration as adult basic education contributes to social inclusion and equity and can play a significant role in helping individuals change their social circumstances (Ranson, 2000; Statistics Canada, 2001). Most educationally disengaged adults report that their disaffection arises and continues as a result of alienating learning environments in formal school situations. Those who fail to achieve at school are more likely to find themselves marginalized by the communities in which they live. They tend to experience reduced opportunities compounded by low levels of self-esteem, motivation, and expectations for the future (NIACE, 2000a; Statistics Canada, 2001). This agency-structure dysfunction is exacerbated by educational accountability practices that 194 inadvertently further social exclusion by replicating and endorsing the same structures that initially precluded these students. For many ABE students the most important initial experience they might have with literacy programs that have an academic foundation and purpose would often be a sense of building self-confidence and social skills. ABE programs are not only concerned with self-improvement through formal education but are also focused on assisting people in their various struggles against poverty and unemployment, discrimination, exploitation, inequalities, and social injustices (Martin, 2000; Thompson, 2001; Upex, 1999). Gains in self-confidence have the potential to assist students in erasing negatively held self-images and better prepare students to make changes in their lives (Upex, 1999). An effective educational accountability framework designed for possibly disadvantaged learners would recognize the academic, social, and psychological diversity of the learners attracted to the program. If policy decisions with regard to funding at the provincial ministry level were grounded in an awareness of the issues surfaced in this study, policymakers might understand that students enrolled in ABE programs often require time to effect real change. How students understand, adapt, and negotiate change within their worlds are elements of personal growth crucial to the students' further achievements in school, life, and work, but these qualities tend not to immediately manifest in test scores and course completion rates. Measures of ABE student success captured through academic outcomes fail to recognize equally significant nonacademic outcomes. Polices that fail to appreciate this could adversely impact adult participation in ABE programs. Policy solutions that favour government sponsored short-term employability programs over ABE are not designed to provide the same social and educational foundations these students 195 need to be successful. Such practices are valuable for their symbolic performance but not for the development of long-term logical solutions to social problems (Scheurich, 1994). From a policy perspective, adult basic education provides valuable social returns through greater equity, income redistribution, and social inclusion. Educational attainment not only benefits the individual by providing better access to higher wages and improved employment opportunities but has been well documented to lead to better health and other favourable outcomes for society (Statistics Canada, 2001). As high school graduation is the absolute minimum educational requirement for today's job-market (ibid.), long-term policy solutions that encourage student success would be a first step toward reducing many social problems that accompany unemployment and underemployment (Ranson, 2000; Evans, 2000; Klasen, 2000). Accountability structures would have the most benefit for adult learners if they resulted in educational policy initiatives that reduced barriers to participation, facilitated further education, and expedited a transition to the world of work (Statistics Canada, 2001). The most successful educational programs for school-leavers appear to have been those that provided a close fit between the policies of assessment, the practices of education, and the needs, interests, and aspirations of adults - that is, when teaching, learning, and accountability processes come together in ways that place the learner's needs at centre stage (Quigley, 1990). Perhaps the most meaningful revelation of this study was the profound impact that educational accountability can have on enabling students to become more successful. Nonacademic indicators of student success are integral to understanding and determining ABE accountability. They, like academic indicators, are significant outcomes of student participation in ABE programs and are fundamental to ensure the efficacy of an ABE accountability framework. 196 Suggestions for Further Research This research offered a contribution toward understanding the multidimensional nature of accountability as it related to adult basic education. Retracing the development of the ABE accountability framework from its inception to demise was intended to provide insights into the political and social dynamics that influenced policy development in a politically charged and ideologically unstable environment. This study demonstrated the complexity of educative accountability and the growing resistance by government to any form of educative accountability that is not systems oriented, and therefore based on standardized and easily quantifiable measures. It also demonstrated the influence social regularities such as governmentality and professionalization (Scheurich, 1994) often have on educational professionals who appear to struggle with an ethical-professional understanding of educational accountability in a world increasingly controlled by a rigid economic-bureaucratic lens. By examining the highly idiosyncratic political and professional context of the development of ABE accountability policy for the school districts of British Columbia, this study revealed wider, more universal themes associated with the power dynamics of professionals, their various governmentalities, and the social construction of failure and success. This has laid the basis for further studies into the social, educational, political, economic, and contextual nature of accountability policies and practices for adult basic education in British Columbia and in other provinces of Canada. Other countries attempting to establish an ABE program to ameliorate the worldwide irony of the "knowledge society's" production of problematic numbers of early school leavers prior to gaining a graduating diploma (OECD, 2006) might find the ideas on accountability presented here useful if applied cautiously and in consideration of their different 197 cultural contexts. Regardless of jurisdiction, given the insights learned in this study, it would be particularly useful for further research to focus on the impact centralized systems-based understandings of educational accountability have on socially constructed marginalized systems of education, particularly on the students and staff within these peripheral systems, and what the long term consequences might be. As I perceived it at the time and looking back now, the school district pilot of the ABE accountability framework, even in its substandard, abbreviated form, gave every suggestion that the mechanism would be a potent process for assisting.ABE teachers, administrators, and government agencies to better understand the complex nature of ABE accountability. It may still be of some use to other jurisdictions, or to British Columbia's school districts at some future juncture. Using the findings of this study as a basis, the construction and implementation of a more equitable accountability system that would be both interperspectival and provisional in nature might begin by conducting further research that solicited perspectives from a greater number of districts and over a longer period of time than the single district's informal five-week review used in this study. This would help to refine the practicability of the framework as well as provide results that would be more illuminating of educational accountability within school district ABE programs. If further research was built on the work offered here, I could not stress enough how greater the likelihood of it gaining acceptance among the auditing minds of decision makers if the qualitative nature of the findings were quantified. Studies concerned with accountability policy development for the public sector in areas other than adult basic education that are similarly engaged with the social and personal development of adults on the peripheries of society and the institutions and systems designed to re-socialize them 198 might also benefit from the approaches and findings of this study. Researchers involved in other forms of nontraditional education, in social work, the prison programs, or in the accountability of institutions and systems attempting to socialize marginalized people more generally, might find some of the themes raised here with regard to systems thinking, governmentality, and the power dynamics of professional groups somewhat comparable or instructive. The process of doing a reflective analysis on a policy initiative has been both gratifying and frustrating. The retrospective case study research tradition provided some emotional distance from the project in terms of time and experience - a sufficient amount to get the project in order and completed with some modicum of calm. Concomitantly, as this research addressed a policy initiative I fervently believed in, one in which I was the principle architect, and of course, as the reader knows, ended in my work being shelved by the current government, the research tradition also has had a tendency to erode enthusiasm as it reinvigorated memory. As it puts the researcher on the line and draws one's humanity and flaws to the surface, it is not an approach to be taken lightly, for the faint of heart or the sake of expediency. 199 References Atkinson, R. (1998) The Life Story Interview. Qualitative Research Methods. Volume 44. Thousand Oaks: Sage Publications. Barton, D. (1994) Literacy: An Introduction to the Ecology of Written Language. Oxford: Blackwell. Beder, H.W. and Valentine, T. (1990) "Motivational Profiles of Adult Basic Education Students". Adult Education Quarterly 40 (2): 78-94. Ben Jaafar, S. and Anderson, S. (in press) Policy Trends and Tensions in Accountability for Educational Management and Services in Canada. Alberta Journal of Educational Research. Bennett, C. (1976) Analyzing Impacts of Extension Programs. Washington, DC: Extension Service, US Department of Agriculture, Report no. ESC-575. Bingman, B . M . and Elbert, O. (2000) I've Come a Long Way: Learner Identified Outcomes of Participation in Adult Literacy Programs. The National Center for the Study of Adult Learning and Literacy. Harvard Graduate School of Education. N C S A L L Report no. 13, February. http://www.ncsall.net/?id=664. Accessed April 2007. Bingman, M ; Ebert, O., and Smith, M . (1999) Changes in Learners' Lives One Year After Enrolment in Literacy Programs. An Analysis of the Longitudinal Study of Adult Literacy Participants in Tennessee. NCSALL Report no. 11, December, 1999. http://www.ncsall.net/fileadmin/resources/research/reportll.pdf. Accessed April 2007. Blackmore, J. (1988) Assessment and Accountability. Victoria, Australia: Deakin University. Cited in Ben Jaafar and Anderson (in press). Block, N . and Stalnaker, R. (1999) Conceptual Analysis, Dualism and the Explanatory Gap. The Philosophical Review 108 (1); 1-46. Boone, J. E. (1985) Developing Programs in Adult Education. Englewood Cliffs, NJ: Prentice-Hall. Boudreau, J.C.N. (1998) Foundations for Implementing an Adult School in a Public School System. Unpublished Master's Thesis, University of Victoria, British Columbia. Bourdieu, P. (1989) La noblesse d'etat (Paris, ed., de Minuit). Cited in Delandshire, 2001. Boyle, P.G. (1981) Planning Better Programs. Adult Education Association Professional Development Series. New York: McGraw-Hill Boyd, W.L. (1988) Policy Analysis, Educational Policy, and Management: Through a Glass Darkly? In Boyan, N . , ed., Handbook of Research on Educational Administration. New York: Longman. British Columbia. Hansard Debates. (2001) 2001 Legislative Session: 2nd Session, 37th Parliament Select Standing Committee on Education. October 26, 2001. http://www.legis.gov.bc.ca/cmt/37thparl/session-2/edu/hansard/el 1026a.htm. Accessed August 2006. British Columbia. Ministry of Advanced Education. (2005) Adult Basic Education in British Columbia's Post-Secondary Institutions: An Articulation Handbook. 2005-2006 Edn. Victoria: M A E . British Columbia. Ministry of Advanced Education, Training and Technology / Ministry of Education (1999) Quality and Accountability. Subcommittee meeting minutes, April 27, 1999. British Columbia. Ministry of Advanced Education, Training and Technology, and Human Resources Development Canada. (1997) Adult Basic Education: Stepping Stones to the Future. Vancouver, BC: The Advanced Education Council of British Columbia. British Columbia. Ministry of Advanced Education, Training and Technology. (1996) Adult Basic Education in School Districts and Colleges. Unpublished internal background ministry paper. May 1996. Archival reference: n:\gill\dave\abe\hist3.doc. British Columbia. Ministry of Education (2001) K-12 Data Reports. Report Number: 1562. http//www.bced.gov.bc.ca/kl2datareports/0 lsldtxt71562.txt British Columbia. Ministry of Education. (1999a) Report on Accountability in the K to 12 Education System. Victoria: Government of British Columbia, http://www.bced.gov.ca/accountability/status.htm. British Columbia. Ministry of Education (1999b) BC Education: Policy [K-12 Manual] Introduction. Victoria: Government of British Columbia, http://www.bced.gov.ca/policy/intro.htm. British Columbia. Ministry of Education. (1999c) The 1999 K-12 Adult Learner Outcomes Survey. Victoria: Government of British Columbia, http://www.bced.gov.bc.ca/adult_graduation. 200 British Columbia. Ministry of Education. (1997) Options for Funding, Delivery and Management of Adult Basic Education. Unpublished MoE document. July, 1997. British Columbia. Ministry of Education and Ministry of Skills, Training and Labour Joint Committee on ABE. (1994) A B E Program Quality Framework. Unpublished MoE document. MoE archive: n:\audrey\abe transitions\quality framework - evalu.'99.doc. Accessed April 2007. British Columbia. Ministry of Employment and Income Assistance (2005) Message from the Minister and Accountability Statement.. September 14, 2005. http://www.bcbudget.gov.bc.ca/2005_Sept_Update/sp/eia/default.htm Accessed August 2006. British Columbia. Office of the Auditor General. (1996) Enhancing Accountability for Performance: A Framework and an Implementation Plan. Victoria: Auditor General of British Columbia. April, 1996. Broheim, S., Bynner, J., Evans. P., Klasen, S., Magrab, P., Ranson, S. (2000) Social Exclusion and Children -Creating Identity Capital: Some Conceptual Issues and Practical Solutions. Paris: OECD, 01/09/2000. http://www.oecd.org/EN/documents/0„EN-documents-614-nodirectorate-no-4-no-19,00.html Bynner, J. (2000) Risks and Outcomes of Social Exclusion. Insights from Longitudinal Data. Paris: OECD, 01/09/2000. http://www.oecd.org/EN/documents/0„EN-documents-614-nodirectorate-no-4-no-19,00.html Canada. Statistics Canada. (2001) A Report on Adult Education and Training in Canada: Learning a Living. May 2001. http://www.statcan.ca and http://hrdc-drhc.gc.ca/arb Cibulka, J.G. (1990) Educational Accountability Reforms: Performance Information and Political Power. Politics of Education Association Yearbook, 1990, pp. 181-201. Coopers and Lybrand (1994) Adult Education A B E Funding Review Final Report. Produced for the Ministry of Education and the Ministry of Skills, Training and Labour, December 1994. Cresswell, J.W. (1998) Qualitative Inquiry and Research Design. Choosing Among Five Traditions. Thousand Oaks: Sage. Creswell, J.W. (1994) Research Design: Qualitative and Quantitative Approaches. Thousand Oaks: Sage. Cross, P. K. (1981) Adults as Learners. San Francisco: Jossey-Bass. D'Amico D. (1999) Politics, Policy, Practice and Personal Responsibility: Adult Education in an Era of Welfare Reform. N C S A L L Report no. 10A, April 1999. Darkenwald, G. and Merriam, S.B. (1982) Adult Education: Foundations of Practice. New York: Harper and Row. Darling-Hammond, 1. (1999) Developing a Professional Model of Accountability for our Schools. British Columbia Teachers Federation Annual General Meeting, March 1999. www.bctf.bc.ca/parents/speeches/darling-hammond.html Delandshere, G. (2001) Implicit Theories, Unexamined Assumptions and the Status Quo of Educational Assessment. Assessment in Education 8 (2): 113-133. Downey, L.W. (1988) Policy Analysis in Education. Calgary: Detselig Enterprises Ltd. Drazen, A. (2000) Political Economy in Macroeconomics. Princeton, N . J . : Princeton University Press. Duffy, K. (1995) Social Exclusion and Human Dignity in Europe. Council of Europe. Cited in Klasen (2000). Eisner, E.W. (2001) What Does it Mean to Say a School is Doing Well? Phi Delta Kappan. January 2001. Elias, J.L. and Merriam S. (1984) Philosophical Foundations of Adult Education. Florida: Robert E. Krieger Publishing Co. Evans, P; Bronheim, S; Bynner, J.; Klasen, S.; Magrab, P.; Ranson, S. (2000) Social Exclusion and Children-Creating Some Conceptual Issues and Practical Solutions. OECD, Social Issues, Equity and Social Inclusion. Paris: OECD. 01/09/2000 http://www.oecd.org/EN/documents/0„EN-documents-614-nodirectorate-no-4-no-19,00.html Faris, R. (1992) Lifelong Learning for the 21st Century - A Report on the Future Development of Adult/Continuing Education in British Columbia. British Columbia: Ministry of Education and the Ministry of Advanced Education, Training and Technology. FPSE (Federation of Post-Secondary Educators) (2006) "Opening Doors and Building Confidence: Proposals to Strengthen Adult Basic Education in British Columbia" January 2006. http://www.fpse.ca/resources/pdfs/0601ABEproposals.pdf Fraser, J. (1997) "Educational Reform and Economic Renewal: Or, It Will Take More Than Robert Reich to Make Schools Work", in Reading, Writing and Justice School Reform as if Democracy Matters, (1-43). Albany, New York: SUNY Press. Fullan, M.G. (1998) Breaking the Bonds of Dependency. Educational Leadership 55: 6-10. Gardner, J. (1990) On Leadership. New York: The Free Press. Gaskell, J. (1988) Policy Research and Politics. Alberta Journal of Educational Research 34 (4): 403-417. 201 Gastil, J.H. (1997) A Definition and Illustration of Democratic Leadership. In Grint, K., ed., Leadership: Classical, Contemporary, and Critical Approaches. New York: Oxford. Glass, G.V. (1972) The Many Faces of Educational Accountability. Phi Delta Kappan. June 1972. 636-639. Grundy, S. (1992) Beyond Guaranteed Outcomes: Creating a Discourse for Educational Praxis. Australian Journal of Education 36 (2): 157-169. House, E.R. (1972) The Domain of Economic Accountability. Educational Forum 37 (1972/73): 13-24. Howlett, M . and Ramesh, M . (1995) Studying Public Policy: Policy Cycles and Policy Subsystems. Toronto: Oxford University Press. Huget, S. (2002) Outcomes of a Province-Wide Consultation: The Big Picture Up Close. Literacy and Learning in BC. Prepared for Literacy BC. Jary, D. and Jary, J. (1995) Collins Dictionary of Sociology. HarperCollins Publishers. Glasgow. Klasen, Stephan. (2000) Social Exclusion, Children, and Education: Conceptual and Measurement Issues. Social Issues, Equity and Social Inclusion. Paris: OECD, 01/09/2000 http://www.oecd.org/EN/documents/0„EN-documents-614-nodirectorate-no-4-no-19,00.html Knowles, M.S. (1980) The Modern Practice of Adult Education. New York: Cambridge. Knowles, R. and Knowles, T. (2001) Accountability For What? Phi Delta Kappan (January 2001): 390-392. Knox, A.B. (2002) Evaluation for Continuing Education: A Comprehensive Guide to Success. San Francisco: Jossey-Bass. Lankshear, C , with Gee, J.P., Knobel, M . , and Searle, C. (1997) Changing Literacies. Buckingham, England: Open University Press. Lauder, H., Brown, P., and Wells, A.S., eds. (1997) Education, Culture, Economy, Society. Oxford: Oxford University Press. Levin, H. (1998) Educational Performance Standards and the Economy. Educational Researcher 27 (4): 4-10. Levin, H. and Kelly, C. (1997) Can Education Do It Alone? In Halsey, A., ed., Phenomenal Changes: Stories of Participants in the Portfolio Project. Durham, NC: Literacy South. Lipman, P, (2006) The Politics of Educational Accountability in a Post-9/11 World. Cultural Studies-Critical Methodologies 6 (1): 52-72. Macpherson, R.J.S. (1996) Educative Accountability Policy Research: Methodology and Epistemology. Educational Administration Quarterly 32: 80-106. Marshall, C. (1997) Dismantling and Reconstructing Policy Analysis. In Marshall, C , ed., Feminist Critical Policy Analysis I: A Perspective from Primary and Secondary Schooling. London: The Falmer Press. Martin, I (2003) Adult Education: Lifelong Learning and Citizenship: Some Ifs and Buts. International Journal of Lifelong Learning, 22 (6): 566-579. Merriam, S. (1998) Qualitative Research and Case Study Applications in Education. Revised and expanded from Case Study Research in Education. San Francisco: Jossey-Bass. Merrifield, J. M . (1998) Contested Ground: Performance Accountability in Adult Basic Education. N C S A L L Report no. 1. Boston: National Center for the Study of Adult Learning and Literacy. Meyer, J.J. (1999) Articulating Learning with EFF Standards. Focus on Basics 3 (C, September, 1999). http://gseweb.harvard.edu/-ncsall/fob/1999/janemeyer.html Mezirow, J. (1991) Transformative Dimensions of Adult Learning. San Francisco: Jossey-Bass Mezirow, J., Darkenwald, G., Knox, A.B. (1975) Last Gamble on Education. Washington, DC: Adult Education Association of the USA. Moore, M.T. and Stavrianos, M . (1995) Review of Adult Education Programs and their Effectiveness: A Background Paper for Reauthorization of the Adult Education Act. Submitted to US Department of Education. Washington, DC: National Institute for Literacy. Moss, D. (2001) Adult Basic Education: Accountability. Unpublished document prepared for the Ministry of Education. National Organization for Adult Learning. (2000a) Young Adult Learners, Disaffection and Social Exclusion. NIACE Briefing Sheet http://www.niace.org.uk/information/Briefing_sheets/YoungAdultLearnersmarOO.html National Organization for Adult Learning. (2000b) Emancipatory Learning. NIACE Briefing Sheet http://www.niace.org.uk/information/Briefing_sheets/EmancipatorylearningmarOO.html National Organization for Adult Learning. (2000c) Student Non-Completion. NIACE Briefing Sheet 3, January 1999. http://www.niace.org.uk. 202 New London Group (1996) A Pedagogy of Multiliteracies: Designing Social Futures. Harvard Educational Review 66(1). Nixon-Ponder, S. (1995). Leaders in the Field of Adult Education: Edward C. Lindeman. http://literacy.kentedu/Oasis/Pubs/0800-l.htm. OECD (2006) Starting Well or Losing their Way? The Position of Youth in the Labour Market in OECD Countries. OECD Social, Employment and Migration Working Paper, Directorate for Employment, Labour and Social Affairs. Paris: Organisation for Economic Co-operation and Development. http://www.oecd.Org/dataoecd/0/30/37805131.pdf. Accessed February 2007. OECD (2002) Thematic Review on Adult Learning: Canada: Country Note. http://www.cmec.ca/international/oecd/adult.note.pdf. Cited in Rubenson and Walker (2006). OECD (2000) The Creative Society in the 21st Century. Paris: Organisation for Economic Co-operation and Development. Pacquette, J. (1998) Equity in Educational Policy: A Priority in Transformation or in Trouble? Educational Policy 13 (1): 41-61. Pietrykowski, B. (1996) Knowledge and Power in Adult Education: Beyond Freire and Habermas. Adult Education Quarterly 46 (2): 82-97. Pipho, C. (1989) Accountability Comes Around Again. Phi Delta Kappan (May 1989): 662-663. Quigley, A. (1998) The First Three Weeks: A Critical Time for Motivation. N C S A L L 2 (A). Quigley, B.A. (1995) Reasons for Resistance to Attending Adult Basic Literacy. Kent: Ohio Literacy Resource Center. Quigley, B.A. (1990) Hidden Logic: Reproduction and Resistance in Adult Literacy and Adult Basic Education. Adult Basic Education Quarterly 40 (2): 103-115. Quigley, B.A. and Kuhne, G.W. (1997) Creating Practical Knowledge through action Research: Posing Problems, Solving Problems, and Improving Daily Practice. New Directions for Adult and Continuing Education No. 73. San Francisco: Jossey-Bass. Rahson, S. (2000) The New Learning for Inclusion and Capability: Towards Community Governance in the Education Action Zones. Social Issues, Equity and Social Inclusion, 01/09/2000. Paris: OECD. http://www.oecd.org/EN/documents/0„EN-documents-614-nodirectorate-no-4-no-I9,00.html Reed and Associates (1998) Success Indicators for Adult Basic Education in The K-12 System: A Report by the Joint Committee to the Ministry of Education. Victoria, BC: Reed and Associates. Robinson, V.M.J . (1994) The Practical Promise of Critical Research in Educational Administration. Educational Administration Quarterly 30 (1): 56-76. Rubenson, K. (1997) Adult Education and Training: The Poor Cousin. An Analysis of Reviews of National Policies for Education. Paper prepared for the Educational Committee Of the OECD. Vancouver and Linkoping: University of British Columbia and Linkoping University. Rubenson, K. (1994) The Foundations and Utilization of Policy-Oriented Research. In Garrison, D.R., ed. Research Perspectives in Adult Education. Florida: Krieger Publisher Company. Rubenson, K and Walker, J. (2006) The Political Economy of Adult Learning in Canada. In Fenwick, T., Nesbit, T., and Spencer, B., eds. Contexts of Adult Learning: Canadian Perspectives. Toronto: Thompson Educational Publishing. Scheurich, J.J. (1994) Policy Archeology. A New Policy Studies Methodology. Journal of Education Policy 9 (4): 297-316. Selman, G. (1977) A Chronology of Adult Education in B.C. Vancouver: Centre for Continuing Education, University of British Columbia. Selman, G. and Dampier, P. (1991) The Foundations of Adult Education in Canada. Toronto: Thompson Educational Publishing. Shohet, L. (2001) Adult Learning and Literacy in Canada. In Review of Adult Learning and Literacy. A Project of the National Center for the Study of Adult Learning and Literacy, Vol. 2., n.p.: Jossey-Bass, Inc., and the Office of Educational Research and Improvement. http.7/www.ncsall.net/?id=558 Spencer, B. (2004) Another Look at "Accountability": Towards a Conceptual Framework for Education Policy Research. Unpublished manuscript. Cited in Ben Jaafar and Anderson (in press). Stagg, D. D. (1992) Alternative Approaches to Outcomes Assessment for Postsecondary Vocational Education. Berkeley, CA: National Center for Research in Vocational Education. Stake, R.E, (1995) The Art of Case Study Research. Thousand Oaks: Sage. Steele, S. (1970) Program Evaluation: A Broader Definition. Journal of Extension (Summer): 8. 203 Stein, S. G. (1993) Framework for Assessing Program Quality. Washington, DC: Association for Community Based Education (ACBE). Sticht, T.G., Armstrong, W.B., Hickey, D.T., and Caylor, J.S. (1987) Cast-Off Youth. New York: Praeger. Stodgill, R,M. (1997) Leadership, Membership, Organization. In Grint, K., ed., Leadership: Classical, Contemporary, and Critical Approaches. New York: Oxford. Street, B. (1995) Social Literacies: Critical Approaches to Literacy in Development, Ethnography and Education. London and New York: Longman Street, B. (1984) Literacy in Theory and Practice. Cambridge: Cambridge University Press, (reprinted 1995). Thomas, A. M . (1983) Adult Illiteracy in Canada: A Challenge. Occasional Paper No. 42. Ottawa: Canadian Commission for UNESCO. Thomas, G. (2006) Review of Adult Graduation (K-12) and Adult Basic Education (Aved) Programs and Services. Final Report submitted to Ministries of Education and Advanced Education, October 23, 2006. Thompson, S. (2001) The Authentic Standards Movement and its Evil Twin. Phi Delta Kappan: 358-362. Torrance, H. (1997) Assessment Accountability, and Standards: Using Assessment to Control the Reform of Schooling. In Halsey, A., Lauder, H., Brown, P., and Wells, A.S., eds., Education, Culture, Economy, Society. Oxford: Oxford University Press. United Kingdom. Ministry of Education and Employment. (1998) The Learning Age: A Renaissance for a New Britain, http://www.lifelonglearning.co.uk/greenpaper/index.htm. Accessed April 2007. Upex, J. (1999) Promoting Adult Student Success in Goal Achievement: A Study of Five Adult Learning Centres in British Columbia. Unpublished Doctoral Dissertation. University of Tasmania. Valencia, R.R. (1997) The Evolution of Deficit Thinking: Educational Thought and Practice. London: The Falmer Press. Venezky, R.L. (1992) Matching Literacy Testing with Social Policy: What are the Alternatives? N C A L Policy Brief, Doc. No. PB92-1. Philadelphia, PA: National Center on Adult Literacy, University of Pennsylvania. Weiss, C. (1991) Policy Research: Data, Ideas, or Arguments. In Wagner, P., Weiss, C , Wittrock, B.,and Wolmann, H., eds., Social Sciences and Modern States: National Experiences and Theoretical Crossroad. Cambridge, England: Cambridge University Press. Wheatley, M J . (1994) Leadership and the New Science: Learning about Organization from an Orderly Universe. San Francisco: Berett-Koehler Publishers. Wiersma, W. (1986) Research Methods in Education: An Introduction. Boston: Allyn and Bacon, Inc. Wrigley, H.S. (1998) Assessment and Accountability: A Modest Proposal. Adventures in Assessment 11 (Winter). Yin, R.K. (1994) Case Study Research: Design and Methods. Applied Social Research Methods Series, Vol. 5. Thousand Oaks: Sage. Young, J. (1999) The Exclusive Society: Social Exclusion, Crime and Difference in Late Modernity. London: Sage. Young, M.B., Fleischman, H, Fitzgerald, N., and Morgan, M.A. (1995) National Evaluation of Adult Education Programs: Executive Summary. Prepared for US Department of Education, Office of the Under Secretary, by Development Associates, Inc., Arlington, VA. 204 Appendix 1 ABE Program Quality Framework, 1994* A B E P R O G R A M Q U A L I T Y F R A M E W O R K DEFINITION OF ABE Note: Adult Basic Education comprises a cluster of programs which have different names in different settings in the Province. The following broad definition has been used by the Joint Committee on ABE. Adult Basic Education (ABE) provides the preparatory courses and skills ranging from basic literacy to provincial and adult secondary school completion and leads to: further education career, technical, and vocational training employability skills employment enhanced personal fulfillment and/or enhanced citizenship skills. A FRAMEWORK FOR ASSESSING ABE PROGRAM QUALITY COMMON COMPONENTS OF A QUALITY ABE PROGRAM Clear mission and goals Adequate, safe facilities Community responsiveness Learner-focused programs Learner assessment Learner support services Qualified staff Opportunities for professional development Relevant curriculum Appropriate instructional strategies Appropriate instructional materials Effective organizational linkages Effective administration Regular program planning and evaluation I Documented outcomes * Source: Ministry of Education and Ministry of Skills, Training and Labour Joint Committee on ABE. (1994). ABE Program Quality Framework. Unpublished MoE document. MoE archivist n:\audrey\abe transitions\quality framework - evalu. '99.doc. Accessed 8/22/2007. Reprinted with permission of the BC Ministry of Education. 205 1. Clear Mission and Goals A quality ABE program has a clearly stated mission and philosophy which defines the program's goals, objectives, and delivery models. . a) The following conditions support the above statement. Please check those which apply to your program. States mission/philosophy and goals, objectives and methods of delivery. Used to promote program to others. Reviewed regularly for relevance by stakeholders. Developed in consultation with stakeholders (people who have an interest in and those who are involved in the program). Observable in practice. Shared with people involved in program. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Five or six conditions met. 3. Three or four conditions met. ' f 2. One or two conditions met. 1. Has no mission/philosophy statement for ABE program. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 206 2. Adequate, Safe Facilities A quality ABE program operates in facilities which are safe, accessible, and equipped to meet administrative, instructional, and program support needs. a) The following conditions support the above statement. Please check those which apply to your program. Space, equipment and technology appropriate for adults and course requirements. Space conducive to learning - safe, free from health hazards. Accommodating to those with a variety of disabilities. Adequate heating, ventilation and lighting. Washroom facilities. Library or space for resource materials and storage. Equipped adequately to meet administrative needs. Private interviewing space. Quiet study space. Space for social interaction and group meetings or recreation. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Eight or nine of ten conditions met. 3. Five to seven of ten conditions met. 2. Two to four of ten conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 3. Community Responsiveness A quality ABE program is rooted in the community it serves. The program reflects its own philosophy and objectives, and it strengthens individuals and their communities. a) The following conditions support the above statement. Please check those which apply to your program. Understands target population the program serves. Systematically collects, analyzes, and responds to community input and feedback. Adapts program to changing population requirements. Is sensitive to the values of the community. Creates opportunities for partnerships. Welcomes the community to events (e.g. Open House) and services. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Five or six conditions met. 3. Three or four conditions met. 2. One or two conditions met. 1. None of the conditions are adequately met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 4. Learner-Focused Programs A quality ABE program respects learner differences and encourages learners to become active partners in the learning process. It encourages a two-way commitment between the program and the learners. a) The following conditions support the above statement. Please check those which apply to your program. Encourages learner participation in as many aspects of the program as possible. Responds to individual learners goals by: an initial personal interview; monitoring progress; providing ongoing support. Shows mutual respect for all those involved in the program. Creates a positive, supportive, safe environment in which learners feel. Comfortable enough to take risks, make mistakes and ask questions. Ensures each learner has a personal learning plan. Makes referrals to other personnel and program options in a sensitive and respectful manner. Assures confidentiality of sensitive information. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Six of seven conditions met. 3. Four or five of seven conditions met. 2. Two or three of seven conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 209 5. Learner Assessment A quality ABE program uses a variety of learner centered assessment procedures on intake, during the program, and at exit. a) The following conditions support the above statement. Please check those which apply to your program. Provides sufficient time to ensure that placement process is thorough. Uses Prior Learning Assessment and accepts foreign credentials where appropriate. Is sensitive to learners' issues (e.g. test anxiety, disabilities, etc.) when testing procedures are used. Uses one or more of the following assessment tools personal or group interviews; journals; questionnaires; group projects; application forms; standardized tests; portfolios; writing sample; activities; written and oral comprehension; competency-based tests. Makes progress and achievement checks at regular intervals within the program. Has a monitoring system for follow-up of learners in place regardless of learner's completion. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Five or six conditions met. 3. Three or four conditions met. 2. One or two conditions met. 1. None of the conditions are adequately met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 210 6. Learner Support Services A quality ABE program has structures and supports in place to ensure equitable outcomes and to help learners from all backgrounds achieve their goals. a) The following conditions support the above statement. Please check those which apply to your program. Identifies special learning needs and arranges for appropriate assistance. Provides intake orientation. Provides advising/counseling for course registration and program. Provides information about support/counseling services within program and community (financial and personal issues e.g. daycare, transportation, etc.). Provides ongoing support for learners. Follows up on learners who do not show for program or referrals. Offers sessions on time management, study skills, learning style inventory and career and personal planning to learners. Offers flexible time schedules where feasible to meet learners' needs. Offers tutorial help for students when needed. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Seven or eight of nine conditions met. 3. Five or six of nine conditions met. 2. Three or four of nine conditions met. 1. One or two of nine conditions met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 211 7. Qualified Staff A quality ABE program employs well-trained personnel with adult education experience, and ensures that instructional assistants and volunteer tutors receive initial training in adult education. a) The following conditions support the above statement. Please check those which apply to your program. Instructors will have a level of education and training needed to address content and process aspects of the program. Instructors will be hired who demonstrate a commitment to their personal and professional development, and model what they teach. Instructors should create a positive learning environment by showing empathy, respect and caring about learners progress. Instructors model and promote student success strategies. All program personnel exhibit teamwork by: meeting regularly; discussing learner's progress; strategizing how to assist learners in meeting their goals. Program provides appropriate initial training and ongoing support for instructional assistants and/or volunteers, based on adult education principles. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Five of six conditions met. 3. Three or four of six conditions met. 2. Two of six conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 212 8. Opportunities for Professional Development | A quality ABE program ensures the provision of a range of support and ongoing opportunities for professional development for all staff. a) The following conditions support the above statement. Please check those which apply to your program. There is designated funding set aside for training/professional development of staff. Staff are given opportunities to keep abreast of current issues and have access to relevant information (publications, videos, etc.). All staff receive training appropriate to the diverse needs of learners. Adequate time is allowed for professional development and meeting time with colleagues in other programs and subject area specialty. All staff make a commitment to create and implement a professional development plan. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Four of five conditions met. 3. Three of five conditions met. 2. Two of five conditions met. 1. One or no conditions met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 213 9. Relevant Curriculum A quality ABE program uses curriculum which is adult-oriented and outcome-based. a) The following conditions support the above statement. Please check those which apply to your program. Curriculum is adult-oriented and accommodates a variety of learning styles. Curriculum outlines are related to program goals, have clearly stated learning outcomes for each course, and established performance standards. Curriculum is reviewed regularly, (internally and through provincial articulation processes). Curriculum is adapted to reflect changing requirements. Curriculum includes a combination of experiential, practical and theoretical approaches. Presentation of curriculum is sequenced to enhance learner success. Program components build on learner experiences and existing skills. Curriculum is laddered to further education and training opportunities. Participants evaluate program components to determine their effectiveness. Curriculum outlines are kept on fde and are available to learners. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Eight or nine conditions met. 3. Five to seven conditions met. 2. Two to four conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 214 1 0 . Appropriate Instructional Strategies A quality ABE program uses instructional strategies which are participant-centered, practical and help adult learners progress towards their goals. a) The following conditions support the above statement. Please check those which apply to your program. Have appropriate contact time for learner success. Have appropriate learner/instructor ratios for learner's needs, level, and mode of instruction e.g. ratios are lower for literacy and special needs. Deliver course content based on adult education principles and utilize the skills and experiences of the participant group. Use instructional strategies which accommodate a variety of learning styles e.g. provide choices in tasks, how to show mastery, use of instructional technology. Help learners assume responsibility for their own learning (i.e. empower learners). Provide adequate staff training and preparation time before any new instructional technology or new courses are introduced. Regard technology as a tool that enhances the learning process and also affirms but does not replace the learner/instructor relationship. Recognize that use of technology helps develop occupational skills and offers communication links with others. Recognize distance learning as an option where appropriate. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Seven or eight of nine conditions met. 3. Five or six of nine conditions met. 2. Three or four of nine conditions met. 1. One or two conditions met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 215 11. Appropriate Instructional Materials A quality ABE program uses a variety of instructional materials which are consistent with the program's philosophy, suitable for adults and relevant to learners' needs. a) The following conditions support the above statement. Please check those which apply to your program. Include appropriate and current technology (e.g. CD-ROM, use of Internet, C.A.I.). Canadian content where appropriate. Encourage meaningful discussion and critical thinking. Create personal meaning from new information and prior knowledge. Relevant and applicable to learner's goals and program philosophy. A wide variety of materials is available and used. Materials are free from bias (except where used to generate critical thinking). b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Six of seven conditions met. 3. Four or five of seven conditions met. 2. Two or three of seven conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 216 12. Effective Organizational Linkages A quality ABE program has effective community and organizational links to help learners move successfully from one educational program to another in order to meet their goals. a) The following conditions support the above statement. Please check those which apply to your program. Hold regular staff meetings to ensure internal communication about program and student progress and easy transition to other training programs. Designated staff regularly meet with representatives of community agencies, including business and industry and helping agencies to share information on program needs, and to help learners in bridging to other career opportunities. Participate in articulation initiatives. Invite community representatives to participate in program events e.g. open houses, graduations. Establish advisory committees where appropriate. Regularly advise/inform community via media of ABE program, student success stories, etc. Develop mentorship programs where possible. Prepare for possible co-op programs and work experience. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Six or seven of eight conditions met. 3. Four or five of eight conditions met. 2. Two or three of eight conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 217 13. Effective Administration A quality ABE program ensures that the program is effectively managed through appropriate policies, procedures and accountability mechanisms and demonstrates leadership in advocating for the student and the program. a) The following conditions support the above statement. Please check those which apply to your program. Appropriate and clear personnel policies in place to address equitable staffing and hiring practices. Opportunities for staff and learners to meet/have input into decision making. Incorporate strategies for recognizing staff achievements, providing appropriate support e.g. professional development. Demonstrates financial accountability consistent with program budget. Adequate time for liaising with funders and community partners. Acknowledges and promotes student and program achievements within institution and to the community. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Five of six conditions met. 3. Three or four of six conditions met. 2. Two of six conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 218 14. Regular Program Planning and Evaluation A quality ABE program regularly engages in program planning and review processes which involve all stakeholders in the program in order to assess its effectiveness. a) The following conditions support the above statement. Please check those which apply to your program. There is a process in place to monitor the implementation of philosophy into practice. Performance evaluation of all program personnel is undertaken at regular intervals. Learners evaluate the process and content of the program at specific times. Learners conduct on-going self-evaluation. Community representatives and professional colleagues have input into the program evaluation process. Evaluations include both qualitative and quantitative measures (e.g. may include satisfaction surveys as well as statistical data). Data are kept regarding numbers of learners enrolled, retention rates, and completion rates of learners. Results of all evaluations are incorporated into program planning and implementation. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Six or seven of eight conditions met. 3. Four or five of eight conditions met. 2. Two or three of eight conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 219 15. Documented Outcomes A quality ABE program clearly identifies, documents and communicates to learners what the intended outcomes are of the ABE program. a) The following conditions support the above statement. Please check those which apply to your program. Clearly stated outcomes are developed for each ABE course and program. Achievement of learners' goals is documented. Learners' knowledge and skills at exit of program are documented. Learners' satisfaction with program is documented. Institutional receiving departments are satisfied with learners' knowledge and skills. Employers are satisfied with learners' performance. Program successes are promoted. A system for long-term follow-up of ABE learners and graduates is set up. b) Given your response(s) in (a), circle the number of the following statement which best applies to your program. 5. All conditions are adequately met. 4. Six or seven of eight conditions met. 3. Four or five of eight conditions met. 2. Two or three of eight conditions met. 1. One condition met. c) Given your response in (b), what steps could you realistically expect to accomplish within the next year to improve in this area? d) What assistance will you require to accomplish the above? 220 Appendix 2 Educational Review Process: Application for Additional FTEs Protocol The review team would like to meet informally with district officials to review the educational review process. This initial meeting would also provide those present with an opportunity to share any concerns. The review team will require a school district designate to facilitate the review process. Process 1. The ministry will ensure that the districts who apply for reallocation of FTE have submitted their MoE/MAETT regional report. 2. A review team comprised of both ministry and field personnel will review the district's adult graduation program and determine eligibility based on the following criteria: 1. attendance records 2. progress reports/student records 3. consistent FTE monitoring, this includes reducing student FTE values if they leave the program, or do not attend/progress on a regular basis 4. learning plans demonstrating how the course work leads to the high school graduation 5. allocation of adult FTE dollars The results of the review will be reported to the ministry in a brief one/two page outline that will focus on the five criteria outlined in the reallocation letter. Each of the criteria will be explored in a brief point form emphasizing strengths and weaknesses in each area and the final narrative will be signed off by the manager responsible for adult education and forwarded to the ADM Educational Programs; a portfolio with hard copy evidence and anecdotal comments from team members will be kept by the manager. The criteria for the one/two page outline will result at the conclusion of review through a discourse process. The review team will determine eligibility for reallocation of FTE in an expeditious manner. Discussion and resolution of their findings will be accomplished through formal meetings where possible, or a conference call process. Any additional FTE's that may be assigned as a result of the review process will occur at re-calc. 221 Janine Wear, as the Provincial Coordinator, will compose the Educational Review Report on the basis of the discourse process amongst the review team members. Seconded and invited Directors who are involved with the review team process will assist ministry staff in understanding the often complicated processes that districts may employ to demonstrate eligibility criteria. They will also assist the ministry staff in obtaining additional information from the district facilitator if there are any questions about the district's ABE practice. Requirements • Room with a phone • Hard Copy Evidence The review team will require the following information on the day of the visit, each team member will require a complete package and the district will be notified in advance regarding the team member make-up. • Financial scan: this should include specific ABE costs and will include personnel costs, supplies, technology purchases (hardware and software etc.), lease costs etc. Shared costs should be broken out to reflect ABE only, i.e., if ABE lease space is one aspect of a larger multi-use complex, the ABE lease cost should only reflect the actual square footage dedicated to ABE. Shared personnel, i.e., clerical, teachers and administrators, should be charged proportionally to ABE. • Provision of ABE teacher timetable, this should include hours of ABE instruction at each location. • Pupil teacher ratio per educational program, i.e., this may vary depending on whether the program is classroom oriented or self-paced continuous intake. • Examples of registration forms, IEP's, tracking process/forms, attendance process/forms. • Address, enrolment, curriculum delivery method (classroom, self-paced, Nautikos, etc.) and hours of operation of each centre • The % of "under nineteen" students who are being amalgamated with ABE students in each centre 222 Appendix 3 Educational Review Team Comments Review Team Comments 1. Attendance. Were students in attendance? If not, why? Ask and check for attendance record verification and assess possible eccentricities. Other comments. 2. Progress. How is student progress kept and tracked. 3. What is process for students who do not attend regularly or who do not seem to be making much progress? Is there a follow-up process? If so, what? Are students who do not attend regularly or make minimal progress reduced (course registrations) for the May 1701 count? What evidence is there of this process? 4. What is curriculum delivery method - i.e., Nautikos (formerly Pathfinder), ILS, teacher driven, classroom, self-paced? Were there a variety of delivery methods in the same complex? How was this being facilitated on your visit? Did it appear successful? Why or why not? 5. Learning Environment. Did the students seem happy and on task? How do they feel about the program? Why? 6. How was staff receptiveness to questions concerning the centre? How well were they able to address any concerns. For example, if there were relatively few students in the centre, what was the rationale? Were the staff able to support their rationale with any form of documentation? For example, if no classes were in session at the particular time did timetables, attendance data and corresponding progress data support justification for there being no students in attendance at that time? 7. If most students are claimed as FTE's how is that supported? Can most of the students attend full time? What evidence is there of that? 8. Is there evidence of counselors and other support staff? 9. Other comments. 223 Appendix 4 Program Assessment The ABE accountability framework's program assessment component was designed in response to issues surfaced during the Provincial School District Review and was intended to be a comprehensive instrument for gathering data on how school districts reported adult learners to the ministry for funding purposes (see the framework at the end of Chapter 4). As described in Chapter Four, the program and student assessment components of the framework were vetted through the Ministry of Education's accountability committee, and subsequently reviewed for value and legitimacy by four rural and urban school districts selected on the basis of being among the most diverse districts in British Columbia. This section provides an understanding of the framework's efficacy at fulfilling its dual motive of flushing out anomalies surfaced in the provincial review while enabling school district ABE programs to justify their actions by gathering data on nontraditional indicators of success appropriate to making a program assessment. As explained in Chapter Five, the school district review of the ABE framework was not able to examine the program assessment component as extensively as student assessment because of the informal review process's abbreviated timeframe. Nevertheless, the school district's ABE program administrator took responsibility for examining the program assessment component for its appropriateness to the district and where relevant included the teachers in this district's insights which are recorded here. 224 Enrolment Questions one to three under the District Level section of the accountability framework were intended to find out, directly or indirectly, how districts reported adult learner enrolments; i.e., whether these were through CE numbers, ministry-funded FTEs, or headcounts. The review district's administrator supported the Provincial Review's preference for school districts to report adult FTEs through a CE number, which was already the district's standard practice. The review district enrolled significantly more full- and part-time adult learners than its reported total of FTEs, which was normal. If a district reported 400 learners and only enrolled 400 learners that would indicate all learners were in full-time attendance, which was unlikely, and might indicate the need for further investigation. Admissions The review district did not consistently use a standardized admissions test common to the district's four regional ABE programs. But as a result of research done by the district's review process, the district chose to adopt the Canadian Adult Achievement Test for enrolling students in the academic grade 11 and 12 curriculum who had not been previously registered with the district's ABE programs. The district followed this route because often students who wanted to enroll in grade 11 and 12 subjects were unsuccessful in their academic pursuits and dropped out if they had not been previously enrolled in the academic upgrading program. 225 Attendance and Progress The administrator found the Facility Level section of the Program Assessment comprehensive in gathering information on regional and geographic conditions that might impact attendance and progress such as a lack of public transportation. The socioeconomic indicators were also found to be effective at eliciting information on learners enrolled in the program. In my view this information would be useful to school districts and the ministry for determining academic and nonacademic indicators of learner and program success, and would also support arguments for understanding program success in terms other than academic course completions. If the framework had been adopted, the review district intended to incorporate the information it acquired into the design of educational pathways that might help learners be more successful. As the framework was not adopted, however, no change in this direction followed. As part of the framework's review, the school district's ABE administrator spent considerable time examining the attendance and progress aspects of program success. The administrator examined the possibility of tracking a number of these indicators, and as a result of conducting the review, developed a tracking system that was able to demonstrate a defensible process for monitoring attendance and progress. (This system is explained fully in the next section.) The philosophy and process of the attendance/progress model that was designed included Individual Education Programs, anecdotal progress reports, and other hard copy evidence that clearly demonstrated student performance as it related to academic requirements. The tracking system was one of the changes to administrative practice that came about as a result of the administrator addressing the suggested data requirements under the framework's Facility Level and Socioeconomic Indicators sections. 226 Student Records The Provincial Review suggested that the effectiveness of the ABE accountability framework would be enhanced through the development of a software program that enabled districts to register students through personal educational numbers (PENs). This would allow the districts to report students electronically to the ministry for funding purposes. The PEN would also correlate individual students with their learning plans, and track and monitor the student's progress in a consistent manner. This information would provide a database that could enable the district to generate student performance reports (incorporating attendance and progress) in conjunction with information relevant to the student's needs, goals, and profde. The administrator found question ten under Facility Level in the accountability framework fairly encompassing at gathering data on how a district tracks students. The review district already had in place a sophisticated software program for tracking students which monitored progress and attendance every 100 hours of possible student contact time, and adjusted FTE values on the basis of student attendance and progress. In my view this tracking process was transparent and would be easy for the ministry to audit. It was compatible with the argument for a standardized school district reporting and tracking system and would be a good initial step to consider province-wide in a ministry-recognized accountability process. If the province were to adopt a similar tracking system and require all school districts to report through a CE number it would eliminate different reporting practices. It would also act as a mechanism to prevent the Casper Syndrome where districts receive funding for ghost students and allocate adult FTE funds to other program areas. 227 The review district's software program allowed for anecdotal comments which enabled a teacher to explicate and legitimize possible inconsistencies in student performance. As part of this district's standard procedures, this data was sent electronically to the district's administrative assistant who, in collaboration with the director, reviewed the 100 hour entries and recalculated the FTE values (when necessary). The recalculation process ensured that student FTE values were current and accurate for auditing purposes and for calculation of FTE values for ministry funding. In my view the software program did a good job of tracking student performance, but was labour intensive. I would recommend the province-wide adoption of a less arduous universal platform that similarly satisfied ministry and student reporting aspects as a preferable means to enhance accountability. "Under 19s" In 1999, when the MoE capped funding for ABE students at the number of FTEs that each district had claimed for the 1998 school year, many districts still experienced enrolment growth and were concerned with the ethical dilemma of turning existing adult students away. This was particularly worrisome when it concerned the high-risk students enrolled in nontraditional school district programs who would not graduate before they turned 19 years of age. In many districts the number of participants under the age of 19 (school age, not adult for the purpose of funding) who were enrolled in nontraditional educational pathways and unable to graduate as school age students far outweighed the number of fundable adult seats. These students could not access adult programs and therefore would not be able to graduate. The "under 19" scenario that the Provincial Review surfaced pointed to a need to somehow accommodate this group of learners as adults. 228 The "under 19" issue did not surface in the review district's accountability process as it enrolled relatively few school age students under 19, and historically had not faced problems with the transition from alternate programs to the ABE program. In my view, the accountability framework would be effective at gathering data on the number of students under 19 who were enrolled in adult centres. It provided information at the district and ministry levels that could support an increase in the district cap which might work to ensure that those students who would not graduate as school age students would none the less be able to access adult programs and have the opportunity to graduate. Literacy Programs vs. English as a Second Language Programs As indicated in Chapter 4, because the ministry does not fund adult ESL students, under circumstances of fiscal restraint it appears that some districts may be prone to confuse literacy programs (which are ministry funded) with ESL programs. This creates district inequities and conflict among district providers of ABE. The Provincial Review suggested that the Ministry of Education might wish to better clarify its expectations with regard to a definition of literacy and ESL. In my view, the data gathered through the accountability framework would assist the ministry in making this distinction. Although the review district enrolled a number of ABE literacy students, in all cases English was their first language; distinguishing literacy students from ESL students was not an issue. Contact Hours The Provincial Review stressed the need for discussion, consensus, and consistency with regard to the number of hours of student involvement deemed necessary for programs and students to 229 achieve success. The Provincial Review found that course contact durations ranged from 80 to 100 hours across the province, with 90 hours being the average. The MoE Accountability Committee was concerned that some students would not be successful in classroom delivery courses such as Math 12, Biology 12, Physics 12, etc., if these were limited to 80 hours, as the students might not be receiving the necessary contact time. This had implications for school district budgets as teacher costs rose with longer contact and instruction times. The review district engaged in considerable discussion about the number of contact hours appropriate for adult learners. The discussion involved all review members, the administrator as well as teachers. The participants concluded that all curriculum would need to satisfy ministry expectations with regard to outcomes, but ministry outcomes for school-aged learners predicated on 100 hours of classroom time to fit the 195 day school year and fulfill the babysitting aspect of K-12 schooling were often too constraining for adult learners. In many courses the allotted time periods for adults to achieve the desired learning outcomes were unnecessary. The teachers thought some courses such as Mathematics 12 might demand 100 hours of contact time or more, while others such as Communications 12 could be delivered in as little as 60 hours. An acknowledgement of these variations would assist the students and programs with course completions, and would ensure the integrity and credibility of the curriculum. Variations in the School Year The MoE Accountability Committee argued for consistency in what constituted a school year at a continuing education centre. The Provincial Review found some school districts closed programs after eight months, others nine, while others operated year round. The variation in 230 what constituted a school year in ABE was problematic on two fronts. First, there was an equity access issue with regard to students having the required time to complete their coursework. This could be particularly important for students who enrolled late in the year and required specific courses for entrance into postsecondary or other training programs for the following academic year beginning in September. Second, if school districts claimed for funding purposes ABE students through a regular school number (see Enrolment section) and limited possible student contact time through an abbreviated school year (e.g., eight months instead of ten), there was not only an equity access issue raised but once again the possible spectre of ghost students (i.e., students the district received funding for who left the program prematurely) and the cash cow stigma sometimes associated with ABE programs. The accountability framework was transparent in its request for data on each district's interpretation of a school year for a continuing education centre. If this data were gathered provincially, it could be used to develop policy to ensure that districts and adult learners were provided with some form of equity access regarding ministry funded adult basic education programs. Quality of Ministry Data At the time of the study, the Ministry of Education did not track adult students by facility or course. Rather than reflecting the courses or schools that adult students were registered in, ministry data only indicated district totals. If the ministry chose to look into an audit process the Provincial Review noted that there needed to be some mechanism to determine where students were attending school and what courses they were registered in. It advised the ministry to take steps to ensure that students enrolled in districts that demonstrated peculiarities with existing accountability processes were actually attending and making progress. The framework could 231 effectively gather data on a number of accountability concerns such as attendance, progress, and course registrations, as it required that each facility record the courses in which adult students were registered. It would enable the ministry to audit adult programs on a facility and course basis as it acted as a mechanism for tracking which schools students were attending and in which courses they were registered. Teacher Qualifications The wording of the School Act at the time of this study had implications for teacher certification. Some districts had certified teachers while others employed non-certified teachers whose professional expertise might have been questionable. This situation was further complicated by the common credential. College teachers were not required to undertake the same certification process as teachers in the K-12 system, yet both were teaching coursework that led to the same credential. This had significant financial implications for district providers, as some districts paid ABE teachers $60,000 a year while others paid $30,000, depending on teacher certification and collective agreements. The Provincial Review pointed to a need to acknowledge and attempt to rectify these district inequities. The framework could effectively gather data on teacher certification. In combination with the budget component of the framework, questions of teacher qualifications would surface district discrepancies with regard to employee costs and teacher qualifications. This transparency would enable the ministry to gauge with some accuracy the quality of academic delivery and could be used to develop a district-by-district funding model. 232 School District Diversity The MoE Accountability Committee recognized that some discrepancy across districts in the academic programs ABE students were enrolled in would need to be tolerated. Literacy and upgrading courses that led to the grade 10 level and courses at the grades 10, 11, and 12 levels that led to a grade 12 graduation certificate would need to adhere to ministry learning outcomes, but the Provincial Review suggested that the purposes of education would be best served if districts had more flexibility to augment existing courses and develop new courses that better reflected the educational needs of the students, their community, and their future career pathways. The Provincial Review found that school districts perceived existing ministry policy regarding curriculum to be overly pedantic especially as it pertained to nontraditional learning pathways such as ABE. The ABE Accountability Committee agreed that districts, in collaboration with their communities, should have greater involvement in determining curriculum and pathways to enhance student success in academics. The issue of school district diversity engaged all members of the school district review. The administrator thought the framework provided the district with a voice for legitimizing the success of its ABE programs, an opportunity to describe its program areas, and a forum for the district to defend its progress and attendance records. Being a large, rural district, there were transportation issues that impacted student attendance and often made it unrealistic to achieve the ministry's attendance policy expectations. All review participants engaged in considerable dialogue around the need for flexibility in developing adult focused curriculum that would better capture the adult learner's life story and still adhere to ministry learning outcomes. All students in the review district were enrolled in a 233 program of studies leading to graduation, which included literacy, upgrading courses leading to the grade 10 level, and courses at the grades 10, 11, and 12 levels that lead to a grade 12 graduation certificate. The data gathered through the discussion process supported the notion for districts to have far more flexibility to augment existing courses and develop new courses that better reflected the educational needs of adult learners. If the framework had been implemented, the district review members resolved to develop more relevant curriculum which might have resulted in argument for changes in ministry policy regarding curriculum as it related to the Adult Dogwood. The review district's administrator recognized that a number of qualitative indicators related to student success could be used to support program assessment reviews. The administrator discussed with the teachers the possibility of further synthesizing nontraditional indicators of student success and using QSR NUD*1ST 6 as a code-based qualitative analysis platform to reinterpret non-numerical data. Given the complex nature of the data the review participants thought this platform might offer the district a toolkit for rapid coding, exploration, management, and analysis of the data. This step might have lent more credibility to nontraditional indicators of success, and, if necessary, the qualitative data could be imported into a quantitative data analysis program such as SPSS to quantify the qualitative in ways more consistent with traditional ministry interpretations of success. 234 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items