UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

Social science, evidence, and social policy : the role of ideology, and the importance of history Bateman, Rebecca Belle 2010

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata


24-ubc_2010_fall_bateman_rebecca.pdf [ 511.91kB ]
JSON: 24-1.0071038.json
JSON-LD: 24-1.0071038-ld.json
RDF/XML (Pretty): 24-1.0071038-rdf.xml
RDF/JSON: 24-1.0071038-rdf.json
Turtle: 24-1.0071038-turtle.txt
N-Triples: 24-1.0071038-rdf-ntriples.txt
Original Record: 24-1.0071038-source.json
Full Text

Full Text

SOCIAL SCIENCE, EVIDENCE, AND SOCIAL POLICY:  THE ROLE OF IDEOLOGY, AND THE IMPORTANCE OF HISTORY   by  Rebecca Belle Bateman  Ph.D., The Johns Hopkins University, 1991    A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF   MASTER OF ARTS  in   The Faculty of Graduate Studies  (Planning)   THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)   June 2010   © Rebecca Belle Bateman, 2010     ii ABSTRACT  Current enthusiasm for evidence-based social policy, which emphasizes the importance of randomized, controlled trials, cost-benefit assessments, and other quantitative methodologies, owes much to earlier periods of history in which social science research was seen as the key to effective policies and programs to combat poverty, homelessness, and welfare dependency.  This thesis contributes to the literature on deliberative policy analysis within planning theory by examining the historical context of evidence-based policy, using the history of poverty knowledge in the United States as an example, to illustrate that underlying ways of thinking about the causes of social issues continue to underpin current attempts at addressing them. Two case studies are presented: current approaches to homelessness interventions, and attempts to break up concentrated poverty through the creation of socially mixed residential neighbourhoods.  In the case of the former, cost-benefit analysis, originally developed in the 1950s to evaluate weapons systems and later applied to the assessment of social programs, has come to define current discourse about homelessness and how most effectively to deal with it.  Social constructions of the “deserving” and “undeserving” poor, nineteenth century concepts, continue to influence the way in which the poor and homeless are viewed.  Ideas about the ill effects of the residential concentration of poverty, which also date to the nineteenth century, inform social mix strategies that seek to stop the “cycle of poverty” through re-development into mixed-income neighbourhoods, and through dispersal strategies such as housing vouchers. Any attempt to analyze how evidence informs social policy must take into account the historical context in which decisions are made, and the resilience of the underlying ideological beliefs and assumptions upon which they are based.  Researchers need to acknowledge the inherently political nature of the knowledge they produce, and embrace rather than avoid the    iii opportunity to subject their interests and assumptions to scholarly scrutiny and debate.  Most importantly, social scientists, through forging closer ties with service providers and their clients, need to recognize and emphasize the value of expertise and knowledge based in practice, and the lived experience of individuals, in the formation and evaluation of social policy.     iv  TABLE OF CONTENTS  Abstract................................................................................................................ ii Table of Contents............................................................................................... iv Introduction ......................................................................................................... 1    Theoretical context and methodological approach .................................. 3 Part One:  What is evidence-based policy?.................................................... 13 Part Two:  Poverty knowledge in U.S. social policy ...................................... 23 Part Three:  EBP and housing policy .............................................................. 53 Conclusions ...................................................................................................... 79 References......................................................................................................... 83        1 Introduction  This thesis is an exploration of the use of social science evidence in the formation of social policy.  Its point of departure is the current enthusiasm for evidence-based policy (EBP), but the discussion goes beyond the present to examine the antecedents of EBP, the historical backstory behind the rhetoric, and some of the consequences of the current association of methodological rigor and evidentiary validity with randomized, controlled trials, and quantitative studies.  The initial inspiration for the topic was a March 2009 report released by the British Columbia Auditor General assessing the province!s homelessness programs as implemented by the Ministry of Housing and Social Development (MHSD).  The report stated that Despite putting in place many best practice strategies and programs, government has not been successful in reducing homelessness.  Clear goals and objectives for homelessness and adequate accountability for results remain outstanding.  Government also lacks adequate information about the homeless and about the services already available to them—this hampers decision-making. Finally, government has not yet established appropriate indicators of success to improve public accountability for results.  I have recommended the development of a comprehensive plan to address homelessness—a plan linking performance expectations to the strategies and programs needed to be successful (Auditor General of British Columbia 2009, 2; 8, emphasis added).  In its response, the Ministry stressed that   Homelessness is an extremely complex issue, and we appreciate the audit team!s efforts to understand both the fundamentals and the nuances.  We also appreciate the audit!s recognition that the Province has initiated a number of best practices and is taking an evidence-based approach to addressing homelessness  (Auditor General of British Columbia, Response from the Ministry of Housing and Social Development, 11, emphasis added).  In anticipation of the public release of the Auditor General!s report, the Minister of Housing and Social Development (MHSD), Rich Coleman, stated that his ministry would begin to “measure the outcomes [of programs serving the homeless] and what people are accomplishing today.”  He went on to say that he would eliminate resources that “are not    2 achieving results” and that his ministry would measure performance of each program to ensure it is properly meeting targets (Vancouver Sun, March 24, 2009, A5). Basing policy on anything other than evidence—whether obtained through research or through data collected by service providers—would seem to be counterintuitive. Yet, the intentional use of the term “evidence-based policy” indicates that provincial decision-makers consider this approach to be worth explicitly mentioning, particularly in the context of responding to the criticism that homeless policy and programs are functioning in the absence of evidence.  This thesis explores the meaning of evidence-based policy, its history and current use, and its particular relevance to social and housing policy.  Of particular concern is the use of social science-based evidence in the development and evaluation of social policy, and the historical, political, and ideological context in which evidence is generated, accepted, and utilized (or not) in policy-making and program development.  The argument presented here is that, using the long history of the use of social science research in the development of social policy (especially policy addressing poverty) in the United States as an example, the current enthusiasm for EBP, in Britain, Australia, and increasingly, in Canada, should proceed with caution.  The distinction of “evidence-based” from “ideologically-based” policy, frequently made by EBP proponents, is a false one, since the belief that evidence can be totally politically neutral, objective, and value free is problematic at best, and faith in the ability of “scientific” data to produce sound social policy itself constitutes an ideological position.  As well, much of the EBP literature either fails to consider, or discounts, the important role of historical context in understanding how knowledge—especially about the poor/homeless—has been socially and historically constructed, and how that body of knowledge “must be assessed as part of historical trends in ideology, politics, institutions, culture, and political economy far more than as a disembodied store of learning about poverty!s [or homelessness! or social exclusion!s] "causes! and "cures!”.  Without this kind of perspective, social science researchers may continue to    3 believe falsely that the data they produce can be neutral or objective when, in fact, research on poverty, the plight of the homeless, housing interventions, and any number of other social issues is, and always has been, an “inescapably political act” (O!Connor 2001, 8; 12).  The discussion that follows is organized into four sections.  Part 1 introduces the concept of evidence-based policy as it is currently used and discussed in the scholarly literature, and as employed by researchers and policy-makers.  Both the pros and cons of this approach are examined, and some of the criticisms of EBP discussed.  Part 2 discusses the history of the relationship between social science and social policy using the United States as a case study. In Part 3, two examples from the housing policy continuum—homelessness interventions, and the use of the concept of social mix in the development and re-development of government- funded housing—are presented to illustrate the differing roles evidence has played and continues to play in influencing policy, and the crucial importance of an historical perspective in understanding the continuities between present and past understandings of social issues and their remedies.   The final section contains the conclusion and a discussion of the lessons of history.  Theoretical context and methodological approach The methodological approach of this thesis is to integrate a number of varied, largely non-intersecting, but related bodies of scholarship—on evidence-based policy, deliberative policy analysis and communicative planning, knowledge utilization, social constructionism, homelessness and housing studies, and social history—in an effort to contribute to a better understanding of the dynamic relationships among knowledge/evidence, power, ideology, discourse, and decision-making in social policy.  The approach is consciously interdisciplinary, and draws on social science and history as well as planning theory.  Its intention is to contribute    4 to the on-going discussion within planning theory on the importance of moving beyond limited definitions of what constitutes knowledge and evidence—often termed positivist, rationalist, or empiricist—by questioning the assumptions of the evidence-based policy approach, and by adding an historical dimension to the discussion of how the meanings of social issues and the characteristics of social groups are constructed. What Yanow (2003, 240-241) terms language-focused methods in the analysis of local knowledge examine the meaning of terms and the use of language among “policy-relevant actors” within a particular context.  But the ways in which people think about and talk about social issues, whether residents of an urban neighbourhood, a planner, or a government policy- maker, are themselves historical products. The sections of the thesis that discuss the social history of poverty knowledge in the United States, and the case studies presented on homelessness and social mix, illustrate how understandings and assumptions about the causes and cures of poverty—and the characteristics of the poor and homeless—combine with an abiding belief both in the power of science to solve society!s problems, and the appropriateness of the use of methods rooted in neo-classical economics, such as cost-benefit analysis, to influence how social issues are understood and discussed.  These understandings and approaches are historical products, and are deeply embedded in the social consciousness. They are often shared among a diverse group of actors in a policy context and may or may not be made explicit, but their ubiquity should not preclude being held up to scrutiny and debate.  Evidence-based policy and the post-positivist critique The term “evidence-based policy,” has a very specific referent:  the use of randomized trials, a methodological model borrowed from medicine and clinical drug trials.  The idea behind EBP is that rigorous evaluations of programs and interventions can connect cause and effect— does a program actually work?—providing solid information that can then be used to inform and    5 guide policy-making. The contention is that the evidence produced is value-neutral, and avoids the pitfall of making social policy based on ideological assumptions rather than on “objective” or “real” evidence. Within communicative planning theory, the idea that evidence, no matter how rigorously produced, can be value-neutral has been discarded, and the emphasis is placed on how evidence actually gets used in practice, rather than on the assumption that the value of information lies solely in how it was produced (scientifically) and its deliberate instrumental use (Innes 1998, 54).  Scientific method is seen a way of knowing, but not the only way, and planning based on scientific style too often leaves out the information most useful to the policy maker—the political ramifications of choices, and the local knowledge of those who know a situation in an experiential and intuitive way.  As Innes (2002, 102) has expressed it, “Without a process for developing a shared understanding of the problem, interactive processes for designing the research co-operatively with the users, and a process for assuring that policy makers are forced to pay attention to the findings of the research in a public arena where they are accountable, the best designed information has little influence.” (Innes 2002, 102). The current enthusiasm for EBP is nothing if not technocratic (i.e., it stresses the role of technically trained researchers/experts and the production of technical knowledge as the basis for decision making and governance [Fischer 1990]), and positivist or empiricist in its underlying assumptions about “best practice” in the production of data for use in the analysis of social policy. Scholars concerned with policy analysis and planning have focused attention on the positivist assumptions behind both a reliance on technocratic approaches to social policy, and the privileging of quantitative/scientific approaches to the generation of evidence used in the development and evaluation of that policy.  Post-positivist, or post-empiricist (Fischer 2003), critiques have given rise to alternative ways of thinking about policy analysis that place more emphasis on discourse, history, narrative, ethics, and participatory approaches to policy    6 development.  Termed “deliberative” or “interpretative,” this approach represents a challenge to traditional positivist attitudes toward knowledge, how to collect data in “a proper way,” and accepted ways of talking about the relationship between knowledge and policy.  Key to an interpretative approach is a belief that a certain conception of how scientific method should proceed—a reflection of its underlying epistemological beliefs—leads to a certain conception of how society should be organized and managed.  Practitioners of deliberative policy analysis have criticized the underlying assumption of traditional policy analysis that objectively generated “scientific” knowledge could provide solutions to a broad range of social and economic problems.  By collecting and analyzing “facts,” traditional policy analysis claimed to be “the voice of rationality” in a contested political world (Fischer 2003; Hajer and Wagenaar 2003; Thacher 2005). Within the social sciences, the positivist paradigm has led to an emphasis on experimental approaches to the generation of evidence that are manipulative, controlling for extraneous variables with the goal of confirming hypotheses (or evaluating the effects of interventions) and based on a belief in the ability of the researcher to maintain a dualist stance in relation to the observed (Guba and Lincoln 1994).  Counter-posed to this quantitative/experimental model is a participatory and collaborative approach that emphasizes methods that prioritize the lived experience of participants (local knowledge), recognizes the role and privileged position of the researcher in the research context, and values all forms of dialogue (focus groups, interviews, storytelling, oral histories, etc.) in an effort not only to gather information, but also to create social change (Feuerstein 1988; Greenwood, Whyte, and Harkavay 1993; Fontana and Frey 1994; Hajdukowski-Ahmed 1998; Lennie 1999, inter alia). Historically, these approaches have been called qualitative, a term used to distinguish a set of field-based, non-statistically grounded methods from quantitative ones, but that also captures the differences of philosophical presuppositions denoted by the positivist-interpretative    7 distinction.  Yanow (2003) prefers to replace qualitative with interpretative since the distinction between qualitative and quantitative has become increasingly blurred—qualitative researchers also count things—and because interpretative underscores the link between methods and their underlying philosophical presuppositions.  How we go about gathering and analyzing data derive from and reflect “prior suppositions as to the reality status and knowability of the subject of our research” (242).  A place for history Both the EBP movement and deliberative policy analysis suffer from a lack of attention to historical analysis. The post-positivist critique of traditional policy analysis asserts a strong relationship between a hierarchical structure of governance—in which a political and economic elite prescribe technologically efficient solutions to what they perceive to be social problems— and a “scientistic” quantitative policy analysis:  the former makes the latter possible, and the positivist methodology used constitutes a political stance.  This is essentially an historical argument, and one that sees today!s less centralized governance structures as providing the context for a more interpretative, pragmatic, and deliberative form of policy analysis and planning practice as opposed to earlier more centralized governance systems (Hajer and Wagenaar 2003, xiii-xiv).  However, as Thacher (2005) points out, an attention to detailed history is lacking in much of the literature in deliberative policy analysis that would support its claims that the fragmented and decentralized nature of governance in modern democracies has transformed the role of policy analysts and planners to facilitators of dialogue among the many actors, public and private, that are in a position to take action on public issues.  Thacher finds this contention unconvincing in the absence of a detailed historical argument, and points out that past eras, such as the U.S. community action programs of the 1960s, also involved a diversity    8 of public, private, and non-profit institutions in the development of policy interventions even though those eras were supposedly typified by a more centralized governance model (455). The resurgence of positivism represented by EBP within “developed democracies of dispersed power, diminishing trust, ambiguous institutions, powerful transnational influences and increasing reflexivity” (Hajer and Wagenaar 2003, xiv) renders the argument put forth by deliberative policy analysts less convincing.  Positivism and scientism, especially in the form of randomized trials, are alive and well both within domestic policy research in developed nations, and increasingly, in the evaluation of anti-poverty interventions by transnational non- governmental organizations in developing nations (Parker 2010; and for a summary see Bannerjee and Duflo 2009). The body of literature on EBP, both supportive and critical, parallels the literature within planning and policy analysis on deliberative/interpretative/communicative approaches—there is very little overlap, Innes (2002) being an exception.  This is unfortunate, because proponents of the evidence-based approach are politically active in getting the model adopted within government bureaucracies, and because a critique of EBP by scholars writing in the deliberative policy analysis field could serve as an important counterpoint to the trend toward establishing the randomized experimental model in a wide range of government agencies, including social policy.  Practitioners of a communicative and collaborative approach to policy development stand the chance of being steamrolled by the EBP movement. While the EBP terminology may trace back only fifteen years or so, the idea that social policy should be based on scientifically produced evidence has a long history.  Recent literature in EBP, however, suffers from the same problem plaguing deliberative policy analysis:  it spends little time looking in detail at past eras that emphasized the relationship between evidence or knowledge and social policy.  As Rothman and Wheeler point out, to ignore historical data “is to miss some of the most illuminating moments in the history of social policy” (1981, 8).  This thesis discusses in some detail Alice O!Connor!s (2001) tour de force historical study of the    9 relationship in the U.S. between social science and social policy in the definition of the causes and cures of poverty as an example of the type of study—assiduously researched and accessibly written—that can be especially valuable in attempting to dig down to the underpinnings of current approaches to EBP, and in explaining the resurgence of experimental approaches to policy research and analysis, particularly in the U.S.  While she writes from within a different body of literature, her discussion is complementary to the work of Frank Fischer, a central figure in deliberative policy analysis, who has written extensively on the relationship between expertise, evidence, and policy, and who has examined in his work eras in U.S. history—such as the Progressive Era, the New Deal, Lyndon Johnson!s Great Society, and the Reagan administration!s neo-conservative backlash against the welfare state of the 1980s—that were crucial to the current position of social science expertise as “a primary medium, if not currency, of modern policy discourse” (Fischer 1991, 348). As both scholars demonstrate, while neo-conservatives were deeply suspicious and critical of the important role played by intellectuals and expertise during the Kennedy/Johnson years, they nonetheless appropriated both the methodological approaches and the data generated by liberal social scientists in order to construct their own network of conservative experts and think-tanks that played a crucial role in restructuring policy discourse in the United States. In O!Connor!s analysis, this led to a shift in emphasis from the War on Poverty!s attempts to alleviate poverty to a focus on ending welfare “dependency” through punitive legislation based on the idea of a “culture of poverty.”  The current emphasis on cost-benefit analysis, which plays an especially important role in studies of homelessness, and experimental method (random control trials), currently advocated as the antidote to “guesswork” in policy- making, traces from these earlier periods, as well. Much of the language, many of the ideas, and even some of the same people were involved in past periods of enthusiasm for research-based social policy as the present.    10 Yanow (2003, 244-45) characterizes the Great Society period as the state developing and implementing policies out of a sense of noblesse oblige, which resulted in paternalistic and usually cosmetic efforts to “empower” communities, as opposed to modern participatory approaches to policy-making that pay more than mere lip service to affording local knowledge “its place as central to the world of professional practice.” However, as O!Connor!s analysis demonstrates, the period of community action research was characterized by the struggle within the Office of Economic Opportunity (OEO) between the promoters of systems analysis, with its emphasis on cost-benefit studies and quantifiable controlled trials, and the community action workers who, though they saw themselves as social “scientists,” nevertheless were committed to local involvement in OEO programs and evaluations, and to reform of social service systems. The main source of contention was which approach would be used to evaluate the effectiveness of antipoverty programs. The systems analysts eventually won this struggle, in part because of political opposition, especially at the local level, to the kinds of activism practiced by community action researchers. The current disconnect between an EBP approach, with its emphasis on quantification and randomization, and collaborative and participatory approaches in planning and policy analysis, illustrates that this struggle is far from over.  It is unclear whether or not policy analysis based on affording local knowledge a central place in professional practice, that recasts the policy analyst from “technical expert” to a “translator, enabling interpretative communities to speak across the cultural divides of meaning and its expressive modes” (Yanow 2003, 245) will fare well when put up against randomized trials and statistical meta-analysis, especially given the politically active nature of some EBP organizations, such as the U.S. Coalition for Evidence- based Policy, in advocating for the incorporation of an EBP approach in all evaluations of government-sponsored social programs.  Despite what some scholars see as a change in the structure of governance from a centralized model to one that is increasingly more decentralized,    11 it is unclear just how much has changed in terms of who has the power to determine how social problems are defined, and how interventions are to be evaluated. And despite the statements of EBP supporters about their commitment to bringing qualitative studies into the evidence-based policy fold, it is currently unclear to what extent that will happen, and whether or not qualitative/interpretative approaches will ever be evaluated on an equal footing with the types of quantitative methods EBP obviously prefers. An historical perspective is important for another reason. As Harvard historian Jill Lepore (2010, 97) points out, “history is hereditary…in this way:  we, all of us, inherit everything, and then we choose…what to disavow, and what to do next, which is why it!s worth trying to know where things come from.”  Analyzing the genealogies of ideas and assumptions is crucial to understanding why social issues continue to be talked about as if they could be solved through the application of the scientific method (often with little or no input from the recipients of interventions), and to an understanding of how particular ways of thinking about socially- constructed groups (the poor, the homeless, the socially excluded) have been inherited from previous eras.  In the two case studies discussed below, homelessness and social mix, an historical perspective helps us to understand from where our ideas about the homeless and how to “help” them derive, and how the categories of “deserving” and “undeserving” have been constructed and re-constructed historically to identify those who are worthy of assistance and interventions, and those who are not.  The current emphasis on cost-benefit analysis in homelessness studies illustrates that that particular methodology and ideology has found favour both with funding bodies and governments, and with activists who now have an economic argument that is often more effective and convincing than a moral and ethical one.  Ideas about the perils of the spatial concentration of the poor and the deleterious effects of their social isolation underlie social mix strategies in a number of western democracies, proof of the tenacity of ideas that have their origins in the mid-nineteenth century.  These are all ideas that we have    12 inherited.  As Lepore said, it is important to know where they come from, and to be conscious of just how much our perceptions of social issues are influenced by them.    13  Part One:  What is evidence-based policy?  Contemporary definitions and rationales   Briefly put, evidence-based policy (EBP) is policy that is grounded in research that has undergone some form of systematic quality assurance and analysis.  Proponents view EBP as an antidote to policy that is driven “more by anecdote, the exigencies of funding constraints, and the ideological perspective of policy makers, rather than a reasoned discussion and review of the evidence” (Culhane et al. 1999, 4-13), policy that can suffer from “frequent lapses into crowd pleasing, political pandering, window dressing and god-acting” on the part of policy-makers (Pawson 2002 quoted in O!Dwyer 2004, 7).  The term originated in the United Kingdom in the field of medical practice, and evolved from the evidence-based medical practice movement of the 1990s that promoted the use of scientific evidence as a basis for professional practice in health and human services.  The movement quickly spread to other countries and to many fields of professional practice, including dentistry, nursing, public health and mental health.  In the field of medical practice, the U.K.-based Cochrane Collaboration has been at the forefront of the push for systematic reviews of all relevant randomized clinical trials in health care, considered to be the “gold standard” of evidence-based practice.  The systematic review has become a cornerstone of evidence-based policy methodology. By 2000, evidence-based practice had been translated into evidence-based policy and had achieved widespread currency in the U.K. among policymakers and academics alike.  The approach spread to non-medical and health-related areas such as social work, education, criminal justice, welfare policy, and urban policy, including housing policy.  The primary impetus for the rapid adoption of EBP in Great Britain was the commitment by the Blair Labour Government to an anti-ideological, pragmatic, and utilitarian approach to “what works” in policy-    14 making:  a conscious and intentional break with the Thatcher years of ideologically-driven policy typified by an adversarial relationship with research, especially social science research. In 2000, researchers established The Campbell Collaboration as a sister organization of The Cochrane Collaboration with the goal, as stated on its Web site, of helping people “make well-informed decisions by preparing, maintaining and disseminating systematic reviews in education, crime and justice, and social welfare.”  The Campbell Collaboration motto—what helps?  what harms?  based on what evidence?—reflects the commitment on the part of EBP proponents to ensuring that policy makers and practitioners do “more good than harm” by making information available that (they hope) can lead to better-informed policy making (Solesbury 2002, 92-94; Marston and Watts 2003, 146-147; Jones and Seelig 2004, 5-6; Chalmers 2005, 228).  Similarly, the U.S.-based Coalition for Evidence-based Policy, founded in 2001, asks what works and what doesn!t work in social policy, undertakes systematic reviews, and stresses the medical precedent of—and is explicit in its preference for—the randomized clinical trial.  The central problem that the nonprofit Coalition for Evidence-based Policy seeks to address is that U.S. social programs are often implemented with little regard to rigorous evidence, costing billions of dollars yet failing to address critical needs of our society, in areas such as education, crime and substance abuse, and poverty reduction. A key piece of the solution, we believe, is to provide policymakers and practitioners with clear, actionable information on what works, as demonstrated in scientifically-valid studies, that they can use to improve the lives of the people they serve.  To address this need, this site summarizes the findings from well- designed randomized controlled trials that, in our view, have particularly important policy implications—because they show, for example, that a social intervention has a major effect, or that a widely-used intervention has little or no effect. We limit this discussion to well-designed randomized controlled trials based on persuasive evidence that they are superior to other study designs in measuring an intervention's true effect (hence their role as the "gold standard" in fields such as medicine, welfare policy, and education) (Coalition for Evidence- based Policy Web site, emphasis added).     15 Growing international interest in EBP has generated a new journal, Evidence and Policy: A Journal of Research, Debate and Practice with the first issue appearing in 2005.  Other collected works on EBP include the edited book, What Works?  Evidence-based policy and practice in public services (2000), a special volume on EBP in Planning Theory and Practice (Vol. 3, No. 1, 2002), and a themed section on EBP and social science research in Social Policy and Society (Vol. 1, No. 3, 2002).  The dedication of whole issues of planning and policy journals to EBP indicates the on-going and robust interest EBP has generated among the developers and implementers of social policy.  Critiques of EBP On its face, evidence-based policy would seem to be a good idea, a positive development in both the formulation of policy and the evaluation of outcomes that would ultimately lead to better-informed, and thus, more effective, policy decisions.  It has, as Tilley and Laycock put it, “all the appeal of motherhood and apple pie” (2000,13 [cited in Marston and Watts 2003, 144]), an intuitive, common sense logic that would be difficult to argue against.  Who would suggest that policy be based on anything but evidence?  Yet, the movement within a diverse number of policy communities toward an EBP-based approach has its detractors.  While none of its critics would argue that evidence-based—also known as evidence-influenced, evidence-informed, evidence-aware, or evidence-driven policy—is an entirely negative development, many do argue that EBP proponents misunderstand the nature of politics and policy, overestimate the capacity of research to inform policy, underestimate the difficulties of forming partnerships between researchers and policy-makers, ignore the evidence of the limited influence of research, and endanger the independence and critical role of academic research in a democratic society (Chalmers 2005, 230;  Jones and Seelig 2004, 9;  Marston and Watts 2003, 146).    16  Marston and Watts (2003) have offered one of the most thoughtful critiques of EBP, expressing an unease with the apparent privileging of some kinds of knowledge, quantitative or controlled clinical trials, for example, over others, such as practice-based experience, professional judgment, and input from end-users of social services.  The reason EBP has become so popular so quickly, they believe, is because of the trend, especially in western democracies, towards a managerial approach to social policy that focuses on cost-effectiveness and efficiency in public programs and service.  Evidence becomes a tool for resource rationing through the determination of which services can demonstrate better value for money.  Social service agencies and non-governmental organizations must increasingly quantify what they do, what works, and why. Far from being the apolitical, strictly scientific enterprise that many of its proponents claim, then, EBP cannot be separated from a broader political context in which the possible and practical take precedence over discussions of justice and interest (Marston and Watts 2003, 149).  Marston and Watts fear that EBP could become a means by which policy elites exert strategic control over what constitutes knowledge and “the truth” by devaluing practitioners! judgments and the voices of ordinary citizens (cf. O!Dwyer 2004, 130). Packwood (2002, 267) sums up some of the concerns about EBP, with particular reference to the field of education. An evidence-based approach to policy is ideological in that it supports particular beliefs and values compatible with the dominant cultural paradigms that define how people and society function.  At present these are determined by definitions of effectiveness as a quantitative measure, professionalism as performativity, teaching as technicist delivery, research as randomized clinical trials, and “credible” evidence as statistical meta-analysis (emphasis added).  With wider public involvement in decision-making becoming increasingly prevalent, the additional question arises of the compatibility of EBP with collaborative policy making.  Planners will be very familiar with this issue, since most decisions regarding planning policies are    17 developed with input gained through participatory planning processes.  Collaborative policy making brings together stakeholders and public agencies to address a particular public issue, and expert opinion is combined with local knowledge of participants as part of the joint fact- finding process.  “Collaboratively developed information is powerful and practical because its meaning is shared, its limitations understood and its future users have a use for it” (Innes 2002, 104).  As Carol Weiss (2001, 291) has written, A movement toward wider participation in policy making will at least demand that researchers, evaluators, and analysts spend more time and thought on disseminating their findings to the relevant publics.  Whispering in the ear of the powerful will no longer be enough.  Researchers and data providers of all types will need to alter their ideas about what it takes to get a hearing for evidence and to master the arts of communication to multiple audiences.  Their findings will have to inform and convince a wide swatch of the public if policy is to be truly evidence-based.  EBP, social science, and the systematic review Echoing the concerns of Marston and Watts, some researchers have expressed an unease with EBP!s tendency, especially through the use of systematic literature reviews, to prioritize and privilege the views of professionals over those of end-users—doctors over mothers in the field of maternal care, for example (Oakley 2000).  The systematic review is a mainstay of EBP, but like many of EBP!s aspects, the field of systematic reviewing methods is highly contested and antagonistic (A. Wallace et al. 2006, 300).  Researchers disagree on how—or even if—systematic reviews of social science research can be done, what “systematic” means, what kinds of studies should be included, and the extent to which literature reviews actually influence policy, among other things (e.g. Hammersley 2005; Chalmers 2005; see also Thomson, Petticrew, and Morrison 2001; and Thomson and Petticrew 2004). Young et al. (2002, 219-220) provide a good overall description of a systematic review.  One distinctive quality of the systematic review is that it starts with a clear question to be answered or hypothesis to be tested.  The reviewer strives to    18 locate all relevant published and unpublished studies to limit the impact of publication and other biases.  [T]he review should explain how studies were identified and obtained, and highlight any known gaps.  In deciding which studies to include and which to exclude the reviewer confronts the issue of selection bias. The quality of these studies is then assessed by examining, in a systematic manner, the methods used in primary studies, investigating potential biases…and identifying the sources of any heterogeneity among the study results.  The results are then synthesised, with conclusions based only on those studies that are methodologically sound….[The systematic review] aims to identify all the relevant research, not just the best known, well promoted and successful.  The systematic review is an important part of the evidence-seeking process and as currently conceived, is an inherent part of the knowledge-driven and problem-solving models of the research-policy relationship (Young et al. 2002, 219).  The main bone of contention among social science researchers, in particular, is the applicability of the systematic review, with its origins in reviews of medical literature, to social science literature.  Medical literature has a limited number of large, high-quality bibliographic databases, which are well-indexed and thesaurus-controlled—and a more exact terminology—while social science literature has a diverse number of databases that are, for the most part, poorly-indexed and abstracted, and the vocabulary within social science is much less consistent and controlled (A. Wallace et al. 2006, 304; Oakley et al. 2005 [cited in Wallace et al.]; Packwood 2002, 269; Young et al. 2002, 222). These features make systematic reviewing within social science more difficult, and for some, ill- advised.  Doing more good than harm…  The debate between Iain Chalmers, former director of the Cochrane Collaboration and a staunch supporter of the systematic review, and Martyn Hammersley, is illustrative of the overall debate about the appropriateness of the systematic review to social science, its role in EBP, and of the role of evidence, generally, in policy-making.  For Chalmers, whose background is in    19 medicine, it is simply irresponsible for policy-makers to “interfere in the lives of other people on the basis of theories unsupported by reliable empirical evidence” (2005, 229).  He prefers the term evidence-informed policy and practice, since he views the link between research and evidence to be indirect, and good research to be essential, though not sufficient, to improve policy and practice.  The systematic review, Chalmers believes, is as applicable to social science as it is to medical science, since all science is a cumulative activity.  In fact, he credits American social scientists with ushering in the modern era of systematic reviews in the 1970s (2005, 232).  As Solesbury (2002, 91) has commented, however, “social science is very bad at the cumulation and the re-use of past research results.” Though he is cautious about making claims regarding the effects of research on policy formation, Solesbury is generally supportive of systematic reviews in the social sciences since the method counteracts the tendency for relevant and often important past research to be ignored.  But both he and Weiss (2001, 290) emphasize that, more than other sorts of evidence, social science data are time-limited and subject to changing contexts, perspectives, and political agendas.  As Weiss says, social science data are not just “out there” waiting to be gathered and analyzed.  Social science evidence has to be “constructed, with definitions, boundaries, data collection methods, analytic techniques, [and] interpretive strategies.”  Hammersley, who comes from a background in education and social science, questions the purpose and benefit of systematic reviewing as opposed to more traditional reviews that are not as time consuming and costly.  He is especially critical of the tendency of systematic reviews to favor randomized clinical trials over other types of studies.  He is concerned that systematic reviewing methodology tends to consider qualitative studies as lacking the scientific rigor necessary to qualify for review (Hammersley 2005, 91-92).  Chalmers (2005, 236-237) rejects the latter argument, pointing to several systematic reviews that combine quantitative data    20 from randomized trials with qualitative data from interviews.  Young et al. (2002, 221) have also pointed out that one of the working groups within the Cochrane Collaboration, the Qualitative Methods Network, focuses on developing standards for identifying, appraising, and synthesizing qualitative research to complement the existing evidence included in Cochrane reviews. In regard to the applicability of systematic review methods specifically to housing research, A. Wallace et al. (2006, 312), based on their analysis of three housing-related systematic reviews, have concluded that “the evidence base that systematic reviews [of housing research] seek to investigate and synthesise is dispersed and heterogeneous, verging on chaotic.”  They suggest that systematic review methods for housing should be distinctive in response to this diversity.  Search methods, contextual analysis, and synthesis all need to be refined, the authors believe, with the nature of the evidence in mind, rather than trying to impose a methodological framework developed for other very different research contexts.  Déjà vu all over again?  How “new” is EBP, really?  Although the terminology and enthusiasm are recent, the idea that policy should be based on evidence, in particular, social science research, has a long history.  Jones and Seelig (2004, 4) trace its roots to the seventeenth century and the Enlightenment, and in Britain and the United States, the origins of professional social science were closely linked to social reform and social policy from the late nineteenth century.  Though, as Solesbury (2002, 93) has commented, current enthusiasm for EBP seems to be “a peculiarly British affair,” at least in its origins, the United States has a long history of a belief in the value of scientific knowledge in solving social problems and informing social policy.  In the area of poverty research, “very much an American invention,” the U.S. has “a degree of specialization and an institutional apparatus that is unmatched in other parts of the world” (O!Connor 2001, 3).    21 Some British EBP scholars have discounted the importance and relevance of the U.S. experience as an example of the influence of social science research on policy.  Nutley and Webb (2000, 31-32), for example, citing an article by Wilensky (1997), accept his argument that even though the U.S. has a strong reputation for policy analysis and evaluation, social scientists have had relatively little influence on social policy, compared to their counterparts in countries with less well-developed evaluation research industries, but closer relationships between knowledge and power.  Wilensky particularly mentions poverty interventions as an area in which social science research has had little effect since the mid-1970s (1997,1252).  However, the history of the generation and use of scientific knowledge about poverty in the U.S. has very important lessons that can and should inform the most recent enthusiasm for “scientific” and “objective” research methods in the development and evaluation of social policy.  The contention that quantitative data, randomized controlled trials, econometric modeling techniques, cost-benefit analysis, and other supposedly politically neutral methods can produce a rational, scientific “cure” for social “problems” is nothing new.  Social scientific knowledge about poverty has played a significant role in the U.S. since the 1970s, both in the definition and framing of the “problem of poverty,” and in its treatment.  Social science researchers may have seen their research used for unintended purposes—developed within a liberal research tradition, but later used by the political right—but it was used, nonetheless.  The history of the relationship between social science and social policy in the generation and use of “poverty knowledge” in the U.S. shows that researchers! contention that the knowledge they were producing was politically neutral, objective, and value-free made it possible for political conservatives to use it to accomplish ends very different from those intended by the researchers themselves.  The U.S. example also illustrates the importance of historical context in the critical analysis of the current enthusiasm for EBP.  Social investigators in both the U.K. and the U.S. have been amassing data on the poor since the nineteenth century in the hopes of creating a    22 body of knowledge that can be used to inform social action.  The belief in the ability of scientifically generated knowledge to solve social problems has long been an article of faith in American liberalism: if we just had enough data—scientifically produced—we could develop policies and programs that “really work.”  Doherty (2000, 180), in the context of the relationship between evidence and housing policy in the U.K., explicitly describes the idea that public policy can be evidence-driven as an expression of the “ideology of scientism.”  Scientism continues to drive efforts to generate and accumulate more and “better” information on a wide variety of social issues related to poverty, including educational achievement, crime, drug addiction, teen pregnancy, and homelessness (O!Connor 2001, 3).  Without an understanding of the roots and tenacity of this belief system, and the fundamentally ideological and political nature of the knowledge that it produces, EBP proponents could perpetuate the mistaken assumption that “what works” and “what!s effective” are value-free, a-political, objective, and empirically determinable “facts” (see also Fischer 2009, 120).     23 Part Two:  Poverty knowledge in U.S. social policy Poverty and social science in the era of Progressive reform In the early 1870s and continuing through the early 1890s, charity reformers in the United States began to re-examine issues of poverty, dependency, and relief.  Originally begun as a protest against the unsatisfactory and costly operation of both public and private relief agencies during the depression of 1873-78, scientific philanthropy emerged as an attempt by reformers to make philanthropy the “science of social therapeutics.”  Promoters of scientific philanthropy were committed to the meticulous collection of statistical information on the poor in the belief that, by gathering detailed knowledge on individuals and families, individualized management strategies could be developed that would treat each case as a unique problem. This new statistical basis for investigations into social conditions was predicated upon a well- developed judgment about the causes of poverty and the predicament of the poor.  While reformers recognized underlying societal and economic causes, most philanthropists developed their diagnoses of the causes of poverty from “the inductive study of concrete masses of dependents, or case counting” which was a natural outgrowth from relief work (Bremner 1956; Ward 1989, 56-60). Charity society workers, some trained in the fledgling fields of sociology and social work (which were virtually indistinguishable at this period), began to learn firsthand that poverty was not just the result of moral failure, but also of economic conditions such as low wages and unemployment, along with poor mental and physical health, and accidents. Philanthropists sought to reform existing policies towards charity for those who did not require institutional care based on new, scientifically based interpretations of dependency that emphasized that poorly coordinated relief promoted and sustained pauperism by providing assistance to those who preferred not to work.  Reformers wanted the State out of the business of outdoor, that is, non-    24 institutional relief, leaving the task of visiting the poor in their homes to private charities.  This approach was also touted as resulting in spectacular cost savings to state and municipal coffers through replacing taxpayer-funded relief workers with voluntary ones (Bremner 1956, 168; Ward 1987, 56; Rothman 1987, 15).  By the second decade of the twentieth century, charitable work had become professionalized in the form of social work, and scientific philanthropy had evolved into the preventive social work movement that was destined to exert a strong influence on the social reforms of the Progressive Era.  In describing the contributions of scientific philanthropy, historian Robert Bremner stresses that the movement, though sometimes strongly criticized for its individualistic orientation, nonetheless fostered the development of a “factual, undogmatic approach to social questions” through the accumulation of detailed knowledge on the personal character and habits, education, earnings, housing, and health of the poor. “The thousands of case records in the files of charity organization societies provided more comprehensive, reliable, and specific data on the actual condition of the poor than had ever before been available to American students,” Bremner writes.  Scientific philanthropy influenced later students of social work to think in terms of preventing rather than merely relieving distress by going beyond the simple withholding of charity to mean the removal of the economic and environmental causes of distress.  By the 1890s, prevention was already coming to be used in this sense by social workers (Bremner 1956, 172-73).  The Progressive Era—the years between 1880 and the beginning of WWI—was a foundational period for twentieth century poverty research.  Poverty researchers emphasized rigorous empirical investigation, quantitative data, and strove to be rigorously objective, as distinct from the morally judgmental approach of charity work.  Researchers were devoted to devising more and better scientific methods for gathering, analyzing, and categorizing the facts of social, rather than merely individual, circumstance.  The knowledge generated was not    25 intended to be neutral, however, but to serve the purpose of reform—looking at prevention rather than relief.  Poverty was “de-pauperized” by redirecting attention from individual dependency to social and especially labour conditions as an underlying cause.  This new poverty knowledge was not entirely without moral judgment:  like the scientific philanthropists, Progressive researchers considered relief a corrupting influence and distinguished between the deserving and undeserving poor.  But researchers also took care to distinguish what they were doing from individual casework; theirs was a study of poverty, rather than the poor.  While rudimentary in its methods and prejudices, Progressive Era poverty knowledge offered at least the elements of a broad framework for later investigations of class, race, and gender inequities in relation to poverty and as problems of political economy (O!Connor 2001, 25; 294).  The social survey  Built upon the foundations of scientific philanthropy, the social survey was used as a tool in the Progressive Era for investigating the causes of poverty, one that blended social science and reform: advocacy through objectivity. The model for social surveys conducted in Chicago, Philadelphia, and Pittsburgh came from a transatlantic source:  Charles Booth!s Life and Labour of the People in London, published between 1889 and 1903 in a series of seventeen volumes. Booth recognized and emphasized the heterogeneity of the poor, and while his survey did not rely on any direct testimonies or household surveys, but on the observations and estimates of amateur investigators, it did collect valuable data on occupations, incomes, expenditures, and housing conditions of London!s working class.  The most well-known of the U.S. settlement house surveys, Hull House Maps and Papers, published in 1895, covered an area in Chicago just to the east of Hull House, and featured colour-coded maps that displayed wage levels,    26 diversity, and residential density.  The survey underscored and illustrated the extent to which race and ethnicity were essential dimensions of social stratification (Ward 1989, 124; O!Connor 2001, 25-29).  The social survey movement made lasting contributions to later poverty expertise through shifting its focus from pauperism to poverty, from the dependent poor to the working class, and from individual behavior as a cause of poverty to industrial capitalism as the main source of economic deprivation.  It laid the groundwork for research techniques that later became essential to “scientific” policy expertise.  One of the social survey!s most important contributions and innovations was the household budget-based poverty line, which served as a scientifically calculated standard of need by which poverty could be measured; the poverty line would be resurrected in the 1960s during the War on Poverty.  Social survey!s use of social mapping and graphic display was intended to energize public opinion, and provided some of the foundations for Chicago School social ecology. Perhaps most importantly, the movement recognized that knowledge gathering was never a perfectly “objective” undertaking in the first place, and that knowledge need not be, or purport to be, “value free” to be legitimate (O!Connor 2001, 44).  Another important, and related, development during the Progressive Era was the rise of professional expertise within fields such as sociology, social work, and education, and the origin of the field of management as a separate professional category. In addition, the focus of the professional became increasingly technical. Professionals and their expertise would come to take on an increasingly important role as policy advisors over the ensuing decades, especially as part of Johnson!s Great Society (Fischer 1991, 336; 2009, 29).      27 The Chicago School and the beginnings of “social science”  As a leader in the movement within organized philanthropy to build a more academic and theoretically grounded social science as a knowledge base for policy, the Laura Spelman Rockefeller Memorial foundation played an important role in subsidizing the work of the University of Chicago sociologists in the 1920s.  The Chicago School took scientific community research in a direction not contemplated in Progressive social inquiry.  Founders Robert E. Park and Ernest W. Burgess wanted to distinguish the Chicago sociology department as a training ground for a social science that was theoretical, experimental, and devoted to uncovering natural laws of human nature and social development. Sociology was established as the science of human behaviour and social psychology, a separate academic discipline from social work.  Low-income “slum” districts of large cities were studied not merely for their own sake, but for what they revealed of human development and human nature generally. Progressives had been committed to policies based on a scientific understanding of social problems, but were too morally judgmental and reform-minded for the Chicago School sociologists.  The accumulation of poverty knowledge became an academic, rather than a reform-minded endeavour, a contribution to theory building, and a scientific basis for a more limited kind of community action.  For all its limitations, Chicago School social ecology provided the tools for an understanding of immigrant and working-class culture that viewed so-called “pathological” behaviours as adaptations to the disruptions of social change.  Social ecology challenged the assumptions of individualized casework, and took the community, rather than the individual or family, as its unit of analysis.  Chicago School sociologists exerted tremendous influence within the discipline, and established the connection between research and policy networks through which their ideas about poverty, social disorganization, and community-based intervention would find their way into local practice, and eventually, into the War on Poverty, three decades    28 after the School!s heyday had passed. As interpreted by a later generation of activists, Chicago sociology provided theoretical grounding for the idea that community empowerment could serve as a vehicle for broader social change (Ward 1989, 153; O!Connor 2001, 44-45; 53-54).  The Great Society and the War on Poverty  When President Johnson made his declaration of war on poverty in 1964, there was still no official definition of poverty.  The nation!s, and the government!s, growing awareness of poverty owed a great deal to journalistic depictions of mass deprivation that began to appear in the early 1960s.  In 1962, Michael Harrington!s The Other America introduced affluent America to the largely invisible world of the poor.  By focusing on the socially deviant characteristics of the poor—though attributing those behaviours to structural causes—Harrington!s analysis was used later to justify a poverty strategy based on individual remediation rather than on political and economic change.  New Yorker writer Dwight Macdonald gave the book an enthusiastically positive review that is said to have caught President Kennedy!s attention. In the structuralist literature, most famously associated with J.K. Galbraith!s 1958 The Affluent Society, analysis was becoming less about the political economy of affluence and a critique of the idea that continuous economic growth would benefit the poor, Galbraith!s argument, and more about the demographic, psychological, and behavioural characteristics that distinguished the poor from everyone else.  This interpretation of poverty was eventually adopted by Council of Economic Advisors (CEA) economists, and absorbed into the liberal growth agenda as a justification for more government spending on the poor.  After a six-month period of intensive planning and interagency negotiation, CEA economists expanded their original strategy focusing on employment and growth to incorporate the structuralist notion of a    29 segregated population with a culture of poverty, and then, the sociological concept of community action through demonstration projects. The CEA was also heavily influenced by three trends in economics that underlay the understanding of the political economy of poverty in the 1960s:  Keynesianism, or at least, its American interpretations, emphasizing the use of government policy to stimulate economic growth and full employment; the post-war revival of neoclassical labour market theory and especially human capital theory, which liberals embraced as justification for social investment in education and training as the key to individual mobility and higher productivity; and the increasing scientism in economics that emphasized mathematical theory and quantification, spurred by expanding computer capacity and sophisticated econometric methodology. In this “new economics,” liberals found a powerful analytic and institutional platform from which to launch the War on Poverty. “The ubiquity of the neoclassical model as a way of explaining the causes and consequences of poverty—alternately labeled human capital, social capital, or cultural capital—indicates the extent to which that central framework still prevails.  So, too, does the overwhelming emphasis on individual-level "causes! of poverty, an emphasis that avoids recognition of politics, institutions, or structural inequality.” Poverty was seen as an intergenerational problem, reproduced through human behaviour, and the new economics introduced a particular approach to studying poverty based on model-building and hypothesis testing, on quantitative and secondary rather than ethnographic and case study research, and on the individual rather than the community as the unit of analysis.  This new political economy of poverty was supposed to be more neutral, scientific, and above all, non-ideological than the old, and was eventually embraced by policy makers as an alternative to the action-oriented direction undertaken by community-based sociological research (O!Connor 2001, 140-143; 146- 156). Social science and economic expertise in the form of policy experts took on a greater role in determining the Democratic political agenda that ever before, playing a very substantial role in    30 both the development of the War on Poverty and the planning of its implementation (Fischer 1991, 338).   By the end of the decade, the U.S government, “through a constellation of sometimes- competing agencies, would develop the most elaborate, far-reaching apparatus for measuring, tracking, and experimenting with programs for poor people in the world.” Staff at the newly established Office of Economic Opportunity (OEO), struck by how little they knew about their enemy poverty, set out to gather the kind of knowledge they felt they needed to win the war: statistically rigorous, methodologically sophisticated, based on nationally representative data, and explicitly modeled on an approach to policy analysis that had revolutionized decision- making at the Department of Defense (DOD) (O!Connor 2001, 139; 166).  The OEO!s Office of Research, Plans, Programs and Evaluation (RPP&E) was the counterpart of the DOD Office of Systems Analysis, the main government location for an approach to policy decision-making developed at the RAND Corporation just after WWII. The RPP&E put this approach into use in the analysis of domestic policy-making.  Systems analysis relied heavily on econometric modeling, originally to evaluate and choose amongst competing weapons systems, that put an emphasis on weighing alternatives in terms of measurable costs and benefits.  The introduction of the Planning-Programming-Budgeting-Systems (PPBS) decision-making technique, which was based on the latest thinking in managerial science, was instituted throughout the government.  President Johnson was enthusiastic about the potential for PPBS to revolutionize decision-making according to scientific principles, and declared it to be the management technique that made possible the War on Poverty. Systemic policy analysis played a key role in shaping the future course of poverty research through its incorporation into the RRP&E and the Department of Health, Education, and Welfare!s (HEW) Assistant Secretary for Planning and Evaluation (ASPE).  Through these government agencies, the protocols of    31 systems analysis were introduced into poverty research (Fischer 1991, 336; O!Connor 2001, 173-5).  Three issues dominated the research agenda of the RPP&E:  poverty measurement, cost/benefit program evaluation, and quantifiable, controlled experiments to test out new approaches to helping the poor.  In the tradition of their Progressive Era forebears, OEO analysts set out to gather precise information on the demographic characteristics of poor people, their income, household structure, and program use, and their likely behaviour over time.  They undertook a series of national surveys, and gathered household and individual-level “microdata” that had a tremendous influence on the scientific understanding of the nature and causes of poverty.  By 1967, Congress was requesting “concrete performance data” and control group evaluations for OEO programs. The RPP&E created an evaluation division to determine whether or not OEO programs were meeting “measurable antipoverty objectives.”  This type of approach relied on concrete indicators and quantifiable outcomes, and impact evaluation concentrated on variables it could measure—such as cognitive ability or income levels, for example—rather than on less easily quantifiable and more elusive indicators of community-level change.  Cost/benefit evaluation was a legacy of the War on Poverty that survived and throve in the post-Great Society years, along with another feature of the RPP&E research agenda—the introduction of a new standard of social experimentation in poverty research.  This new standard would consist of demonstration projects, conducted under “reasonably controlled conditions” that would be replicated in several sites, and based on experimental design through the use of randomly assigned control groups. Academic researchers affiliated with specialized research institutions conducted the demonstration research. They tested the impact of policy changes on individual behaviour—these were not tests of theories of empowerment and systems change (O!Connor 2001, 182-190).    32 Science v. community action  For the first part of the OEO!s seven-year lifespan, two approaches to fighting the war on poverty coexisted, though uneasily: the politically controversial Community Action Program (CAP) and the RPP&E.  However briefly, the OEO, in its earliest inception, cultivated a more overtly political brand of action research.  It was only with the eventual political demise of community action that the analytic, scientific model came to dominate OEO-sponsored poverty research (O!Connor 2001, 167).  The CAP drew its intellectual base from a series of community-based experiments in the 1950s and 1960s sponsored and funded by the Ford Foundation, the National Institute of Mental Health (NIMH), and the President!s Committee on Juvenile Delinquency.  Like their Progressive Era counterparts, these experiments treated low-income urban neighbourhoods as laboratories for research-based reform that was seen as an on-going process of knowledge creation and mutual understanding involving expert and “indigenous” collaboration.  Unlike the Progressive Era, community action researchers set themselves apart from social work and aspired to systems reform. Community action had more to say about cultural adjustment and social organization than about employment, wages, and work conditions that were the basis of Progressive social surveys.  These were mostly male academics and foundation experts, and their efforts were more consciously scientific and experimental (O!Connor 2001, 124-125). (Along with the professionalization of social science and the increasing emphasis on objectivity, theory, and hard data, had come a tendency for women researchers to be pushed to the periphery, and the denigration of much of their Progressive Era community research as “soft- hearted do-goodism” [Weiss 1995, 139]). Resident involvement was a key part of community action programs, along with the idea that programs should increase a community!s “competence”—its ability to deal with its own    33 problems without reliance on outside experts.  Local participation and indigenous reform later became highly controversial within the politicized context of the War on Poverty. Urban planners were also exploring the possibilities of community action as a vehicle for comprehensive systems reform. Along with mayors and journalists, planners were concerned with the large- scale changes in the structures of American cities in the post-war era:  industrial decentralization, white middle-class suburbanization, and the steady stream of migrants that included poor whites from Appalachia, southern African Americans, and Latinos who were moving into old immigrant urban neighbourhoods. While also concerned with “bricks and mortar” issues of urban renewal and issues of urban governance, planners were coming to grips with the new human face of urban reality through the development of programs to help the new migrants assimilate to urban life.  In community action, planners saw a means to achieve a comprehensive, integrated approach to providing services, and a platform for encouraging broader “systems reform.”  Systems reform was informed by Chicago School concepts of migration processes and urban ecology, but was even more the product of newer and more applied disciplines like urban planning and public administration.  The vision of systems reform originated not in academic settings or urban experiments, but in the context of the post-war struggle for urban renewal and redevelopment. Community action saw social science as a change agent, and the knowledge produced was to be based on a hands-on, activist, and practical as well as theoretical approach.  To its supporters, community action was political as well as social learning, and they advocated for its inclusion in federal policy.  Community action was actually based on a fairly conventional ideal of the poor, though it emphasized community and environmental rather than individual analysis. Rather than analyzing structural inequalities that produced poverty, community action stressed concepts such as social deprivation, community disorganization, and the inadequacy of social service systems.  Even the most “bottom-up” of the experiments started out with a somewhat    34 apolitical idea of rational, comprehensive planning as the essential approach.  These features made community action more politically acceptable to the Kennedy and Johnson administrations, but they also came under challenge in 1961-63 when the experiments actually got underway.  Rational systems reform met with a great deal of political and bureaucratic resistance, and because resident participation was a rather vague concept, some, like mayors and planning experts, saw participation mainly as a way to gain consent for centralized redevelopment plans, while others had more radical ends in mind, such as the confrontation of racism (O!Connor 2001, 124-136).  “[C]ommunity action!s experimental idealism—its belief that change could be achieved in a scientific process of planning, experimentation, assessment, and knowledge application—was itself being challenged by the incompatibilities between the scientific objectives of research, and the political demands upon action.”  Political demand for quick and visible results required time frames that were much too short—no community-based project could be expected to produce results within two years.  Proponents needed a more powerful institutional base from which to launch community action as a instrument of reform and social policy, and they found it in 1963 when officials in the Kennedy and Johnson administrations geared up for the War on Poverty (O!Connor 2001, 136).  Once incorporated within the OEO, conflicts arose between OEO analysts, with their emphasis on quantification, measurement of goals, outcomes, and costs, and controlled experiments, and Community Action Program social theorists over who should determine the standards for evaluating the success of antipoverty programs.  As O!Connor observes (and as most community-based service providers would agree), “the problem…was that community action did not lend itself to such measures at all:  it was about process as much as product— intangibles (or, at least, unquantifiables) such as empowerment, self-determination, and quality    35 of life….[T]he "input and output! indicators imposed by outside evaluators were anathema to what the idea of community action was all about” (O!Connor 2001, 186). Community action had always held at least the potential to challenge authority and to serve as an instrument for more radical reform.  Despite this potential, by 1964, community action had become primarily aimed at changing the cultural deficiencies of the poor based on the idea of the culture of poverty.  Poverty was, by then, defined as a problem of individual deprivation rather than the result of systemic social inequality and economic change.  This, in turn, disconnected poverty from broader issues of urban restructuring and urban politics by pathologizing it. The War on Poverty heightened and exacerbated the latent conflicts within community action—between a broader vision of community reorganization and institutional reform and a narrower effort to “break the cycle of poverty”—ultimately resulting in its demise (O!Connor 2001, 159-64). Community action also ran up against unexpected criticism and resistance within the communities themselves.  As Fischer (2009, 35-36) points out, many of the communities within which the activists/professionals were working were more interested in the technical knowledge the activists brought with them, and less interested in having outsiders advocate their causes for them.   Roundly denounced by their professional organizations for criticizing the technical knowledge and practices of their fields, and themselves downplaying their expertise in order to become more community organizers than professional outsiders, community action workers lost their basis of legitimate authority.  Without it, they were just another group of political activists seen to be competing with the “indigenous” activists.   While the Nixon administration vowed to dismantle the War on Poverty, government demand for analytic poverty knowledge showed no signs of slowing down.  Indeed, from 1965 to 1980, federal funding for poverty research rose from nearly $3 million to just under $200 million, most designated for the applied purposes of measurement, program evaluation, and policy    36 analysis.  Federal dollars facilitated the development of new and sophisticated methodologies for carrying out these tasks, with the result that poverty research had an impressive arsenal of survey, experimental, and modeling techniques unmatched in any other field of social research.  As part of the emphasis on experimental design in the development of social programs, the administration established the Manpower Demonstration Research Corporation (MDRC) in 1974 to sell the employment and training field on the concept of experimental design in demonstration research. Long seen as the road to social rehabilitation and self-sufficiency for the most recalcitrant of the poor—nineteenth century work requirements for the able-bodied residents of almshouses were an antecedent—the Supported Work demonstration project was designed to help clients acquire the discipline and personal habits necessary to gain and hold a job through the provision of transitional jobs.  The program had a commitment to the idea of rigorous, unbiased, and policy-relevant research through the use of experimental methods that favoured the random controlled trial—eligible applicants would be divided into “treatment” and “control” groups.  Evaluations of Supported Work concluded that the program showed promise, and helped to establish legitimacy for the idea of work-based approaches to welfare reform.  Once again a journalist, this time it was Ken Auletta in a series of articles about participants in the program and published under the title The Underclass (1982), publicized the program, and especially his characterization of the participants as criminals, drug addicts, long-term welfare recipients, and high school dropouts.  These features came to identify an undifferentiated group of poor people in what was rapidly becoming the new terminology for the undeserving poor (O!Connor 2001, 231-237).  The MDRC was a standard-setter in social policy demonstration and policy-related research in such areas as public housing, work and welfare, youth unemployment, and teen pregnancy, primarily with financing from government contracts.  But as the requirements for    37 demonstration research became more strictly experimental, they also became less associated with social activism and reform (O!Connor 2001, 236-37). Judith Innes has characterized this period as “an evaluation fad” during which social scientists involved in program evaluations “looked for quantitative measures of programme outcomes against predefined goals and all too often missed the actual value of the programmes to people in them.”  “Most of these efforts,” she observed, “in focusing on "objective data,! neglected many other types of knowledge, including qualitative research, experiential, intuitive knowledge and stories [that] were often dismissed as anecdotal and unscientific” (2002, 103).  Carol Weiss, who was involved in program evaluations in the late 1960s and 1970s, has described the era as “the high point in social science for policy” when “many of us [social scientists] believed that a rich country could for the first time arrange its affairs so that the poor were no longer with us.”  However, she judges the demonstration experiments as largely ineffective.  “By and large,” she writes, “the schemes were shelved” (1995, 140; 2001, 284).  Out of this period arose a body of research on knowledge utilization that was rooted in researchers! frustration with the apparent lack of attention to their research results.  Scholars began to explore how policy-makers used social science research, and what effect it actually had on which programs were adopted, how programs were operated, or whether modifications were made. Researchers involved in program evaluation found that although research was requested and funded by a government agency, there was often a puzzling lack of response to a report!s conclusions.  Even when evaluations of some 1970s-era job training programs disclosed shortcomings, the programs continued to be funded, anyway (Weiss 1995, 140-41). Rossi offered this explanation for the dilemma: Thus it turns out that one of the major obstacles to evaluation research is the interest in maintenance of a program held by its administrators.  Their ambivalence is born of a two-horned dilemma:  On the one hand, research is needed to demonstrate that the program has an effect; on the other hand,    38 research might find that effects are negligible or non-existent (Rossi 1972, 227 quoted in McAfee 1975, 79).   McAfee (1975) in an evaluation of mid-1970s social science-based evaluative research concluded:  [E]valuative research can be characterized as lacking in content, integration, and relevance to policy concerns, ignoring social costs, stressing efficiency over effect, seldom considering second order impacts and containing such methodological and assumptive weaknesses as to leave one confused as to whether it was the programme or the research which was defective (82).  Weiss (1995, 148) made a similar charge that much policy-oriented social science research was mediocre, atheoretical, took a short-term view, ignored earlier research on a topic, focused on only a subset of the variables that were operative (often the ones that could be quantified), used obsolete methods, or made recommendations far beyond what the data would support.  She called for more rigor and higher quality in research, and pointed out that the credibility of research for policy development is compromised when a variety of studies of the same issue produce widely divergent results.  This serves mainly to confuse rather than enlighten policy makers—is it the policy, or is it the research that is the problem?—with the result that “committed partisans are inclined to find the study that agrees with their position and stick by it” (1995, 148).  Weiss identifies four I!s that new data or research findings confront—ideology, interests, institutional norms and practices, and prior information (2001, 286).  Ideology refers to people!s basic values, the “filter” through which evidence is passed and selected before it impacts on policy (Doherty 2000).  Evidence that conforms to an individual!s or government!s current ideological stance will be accepted and cited; countervailing evidence will be discredited or downplayed.  “Governments and policy makers can screen out certain perspectives.  Ideological filters, or views on how a society should operate, can prevent politicians from receiving views    39 inconsistent with their own” (Bryant 2004, 639).  The result is that “failed policies can remain in place because they suit the prerogatives of established interests, though they may have little evidentiary basis for continuing” (Culhane et al. 1999, 4-13).  As valid as these observations are, O!Connor!s study of U.S. poverty knowledge clearly shows that the research itself was not apolitical, counter to the claims of those who produced it. In the case of the MDRC demonstration programs, the research was devoted to already politically sanctioned programs, and to what lent itself to testing with quantitative, analytic techniques (O!Connor 200, 236).  Since the research was largely government-funded, researchers were required to “accept the assumptions of officialdom, look for "feasible! solutions and give up their true role in society, which is that of critic of social and political institutions” (Weiss 1995, 138-39).  And as the later history of its use demonstrated, the body of knowledge produced during the 1970s heyday of social science policy research, though in many instances produced within a liberal research tradition, would take on a new life within a very different political context in the 1980s and 1990s.  With the beginning of the Reagan administration in 1980, poverty research institutes and evaluation programs were gutted.  Like his conservative counterpart, Margaret Thatcher, Reagan!s administration rejected “value-free” policy planning in favour of an explicitly—and proudly—ideological approach to social policy, one that was hostile, in particular, to social science research (O!Connor 2001, 244-45). The neoconservative political response to the liberal reform agenda, and the role of expertise in formulating and implementing it, was to reach out to their own conservative brethren in the intellectual world, especially those in academe. Throughout the 1980s, corporate business leaders and conservative experts established a working relationship that resulted in the founding and financing of a multimillion-dollar network of conservative think-tanks.  Though highly critical of the role of liberal scientific expertise in influencing policy during the Great Society years, neoconservatives nevertheless adopted the    40 very same approach, though the locus of their reform activity was in a growing network of foundations and institutions outside of formal government institutions, rather than the White House (Fischer 1991, 341-342).  The War on Poverty becomes the war on the poor The Reagan administration!s focus would not be on ending poverty, but on reforming welfare, and the programs of the Great Society were rejected as having actually made things worse for the poor by increasing their dependent status. One of the most influential proponents of this view was Charles Murray, a conservative social scientist whose book, Losing Ground, published in 1984, used aggregate statistics on poverty, originally collected by liberal analysts, to argue that welfare state social programs had increased the dependency of the poor on government benefits, and had undermined the work ethic and the family.  His solution was to eliminate social welfare programs, except for unemployment insurance, for everyone but the elderly in order to push the poor to learn to fend for themselves (i.e., attain “self sufficiency”).  In emphasizing the deleterious effects of welfare, Murray was echoing the scientific philanthropists! mantra of “not alms, but a friend:”  the assertion that what the poor really need is to be encouraged to develop “self-respect, hope, ambition, courage, character,” not undeserved handouts (Bremner 1956, 171; O!Connor 2001, 244-245). Murray made his argument in a simple and accessible way, with a tone of detachment common to most analytic research.  Critics pointed out that the statistics Murray used actually showed that the poor had benefited from the growth in government transfers in the 1970s, and that, contrary to one of Murray!s main points, statistical analysis showed no link between social programs and an epidemic of out-of-wedlock births.  But the nuances of the methodological errors the critics pointed to were largely lost on the average reader.  Murray used the fictional    41 couple Harold and Phyllis to show that, however detrimental to themselves and to society at large, the kinds of choices poor people made—which included choosing welfare over paid work, and illegitimacy rather than marriage—were actually rational choices, given the incentives of welfare.  Thus Murray used the same rational choice logic that liberal poverty experts had done much to promote:  poverty knowledge turned out to be not as ideologically neutral as it made itself out to be (O!Connor 2001, 248-49).  Reagan!s Assistant Secretary for Planning and Evaluation (ASPE) set out a research agenda that made welfare dependency and its prevention the administration!s chief concern. Eager to diversify the poverty research network, the ASPE funneled big dependency projects to the John F. Kennedy School of Government at Harvard, among others.  In 1983, economists Mary Jo Bane and David Ellwood from the JFK School produced a paper on welfare dependency as part of a larger project financed by the ASPE.  In it, the authors developed a new approach to measuring dependency that instantly transformed the received wisdom, developed after a decade of analysis, that poverty and welfare were transitory conditions, amenable to solution with a better system of income support. Bane and Ellwood!s study, using the same data and a different methodology, concluded that dependency, rather than the exception, was instead the rule, and a much bigger problem than it first appeared.  What they called “long-termers,” who tended to be never-married mothers, high-school dropouts, and Black, constituted a minority of welfare recipients, but represented the majority of recipients at any given time.  Along with “recidivists” (a term that tended to equate welfare recipients to criminals), long-termers represented nearly two-thirds of the costs of welfare.  “Short-term” recipients tended to be white, divorced, and relatively well- educated. By identifying family break-up as an independent variable causing poverty, Bane and Ellwood helped to confirm popular suspicions that poverty was less an economic problem than a symptom of disturbing new demographic and behavioural trends (O!Connor 2001, 251-53).    42 Stereotypes of welfare mothers, especially Reagan!s own oft-repeated, though bogus, story of the Cadillac-driving “welfare queen” exploited the white backlash against the civil rights movement, and managed to attack policies targeted towards African-Americans and other minorities without specifically mentioning race (Krugman 2007).  The undeserving poor became associated in particular with people of colour and especially unmarried women with children living in inner-city ghettos.  William Julius Wilson!s 1987 book, The Truly Disadvantaged, described an urban “underclass” whose behavioural pathologies, though in his analysis, a response to structural conditions such as long-term joblessness brought by urban deindustrialization, nonetheless came to represent the undeserving, dangerous, pauperized poor. The notion of a culture of poverty rooted in behavioural deviancy continually threatened to undermine Wilson!s more structural analysis of ghetto poverty, especially in early efforts to implement his theory within the poverty research industry.  The ASPE issued a notice shortly after Wilson!s book was published calling for theoretical models linking underclass behaviors (drug abuse, teen pregnancy, welfare dependency) to so-called “neighbourhood effects” such as what nineteenth century “epidemic” theory referred to as a process of “immoral contagion” in poor neighbourhoods (O!Connor 2001, 267-272).  (Wilson!s work would later come to influence housing policy in the form of attempts to achieve “social mix” and thus uplift the poor through the provision of working residents as models of the benefits of employment.)  Analytic research played an extremely important role in the rise of workfare during the Reagan administration.  In 1986, the MDRC began to release its findings on evaluations of state-level work and welfare programs that it had initiated in the aftermath of the first round of budget cuts.  While the findings were not all that impressive in terms of the success of the programs, they were nonetheless held up as evidence that it was at least feasible to tie welfare to work and training obligations, especially for welfare mothers.  What made the MDRC findings    43 so influential was the organization!s reputation for rigor and neutrality, its ability to package findings in accessible policy briefs, and its emphasis on experimental design.  The MDRC!s work has been hailed as a major success story in the annals of policy research, and combined with Bane and Ellwood!s work on dependency, formed the most important factor in galvanizing a legislative welfare consensus behind the Family Support Act of 1988.  The Act required that absentee fathers take full financial responsibility for their children, and that all able-bodied mothers be expected to work. The Act also contained provisions allowing states to apply for federal waivers so they could experiment with new approaches to getting people off of the welfare rolls and into jobs. By insisting that their research was politically and ideologically neutral, liberal policy analysts played an active role in making family structure and dependency the main issues in poverty policy.  Their research, largely funded as it was by government dollars, reinforced rather than shaped federal policy while failing to provide an alternative analysis that stressed employment, wages, and growing inequality that were consistently emerging in their own research as the underlying causes of poverty (O!Connor 2001, 258-59; 289). With the election of Bill Clinton in 1992, liberal poverty researchers were hopeful that the anti-research era of the Reagan-Bush years was over.  Clinton was a proponent of knowledge- based policy and reform, and he appointed David Ellwood and Mary Jo Bane as the Health and Human Services assistant secretary for planning and evaluation (head of the ASPE), and assistant secretary for children and families, respectively.  Weiss, writing in 1995, pointed to Bane!s and Ellwood!s appointments as evidence that the status of social science research was “on the upswing.”  However within a year, Ellwood had left to return to Harvard, and Bane had resigned in protest over Clinton!s approval of a bill touted as “ending welfare as we know it” (O!Connor 2001, 284-85; Weiss 1995, 146).    44 The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 was signed over the objections of the experts Clinton had invited to design welfare reform.  The bill did away with the New Deal guarantee of assistance for families with children by imposing strict work mandates and time limits on welfare eligibility, and by radically devolving responsibility for poor relief to the states.  Ellwood, with support from other poverty experts, had been proposing an end to welfare since 1988, but his version was far different from the legislation that Clinton signed.  Ellwood!s plan had included an expanded earned income tax credit (essentially a negative income tax targeted to the working poor); a higher minimum wage; and broader health care coverage to ensure that work would be a better-paying alternative to welfare.  His vision of the end of welfare would also have included temporary income assistance combined with education, training, job placement, and work experience assistance; a government guarantee of child-support payments when absent parents could not pay; child care; medical coverage; and transportation assistance.  At the end of two or so years of transitional assistance, recipients would either be cut off welfare or required to accept a government job at minimum wage.  These came to be known as “soft” time limits, because Ellwood!s plan did not include cutting people off from all forms of public assistance, as the signed version of the law prescribes.  The bill that was unveiled in 1994 was a much stripped-down version that lacked any mention of government child support assistance or guaranteed jobs.  Additional spending was to be financed by cutting other social programs such as Food Stamps, and immigrant, emergency shelter, and disability benefits. In 1994, radically conservative Republicans under the leadership of Newt Gingrich swept the mid-term elections and took control of Congress.  The enthusiasm for welfare reform was pushed even further to the right.  But, as O!Connor argues, “the seeds of this rightward pressure had been built into the logic of mainstream social scientific reform.  That logic…had defined long-term dependency and individual behavior as the problem, effectively dismissed any notion    45 of legitimate entitlement in favor of individual responsibility, and…had subordinated the goal of fighting poverty to the more politically palatable welfare reform” (2001, 290).  It was not the case that poverty researchers failed to focus on other causes, such as wage stagnation, income inequality, industrial restructuring, and racial discrimination.  The problem was that the poverty research establishment, throughout the Reagan/Bush years, had put more energy into studying, evaluating, and experimenting with welfare than into strategies to reverse the growth in inequality, restore full employment at higher wages, stop low-end labour market decline, or to study ways to create jobs for all the welfare recipients they expected to join the labour force. The poverty research industry had developed a dependency problem of its own as it responded and conformed to the changing political climates and agendas it relied on for funding, rather than establishing and working to gain support for an independent policy agenda to deal with poverty at its roots. By claiming to be analytically neutral, the data policy researchers produced could be, and were, appropriated by conservative politicians to support their arguments for welfare reform. The fact that a deeply conservative political regime incorporated a redefinition of poverty in terms of dependency and individual pathology into a punitive law did not, as O!Connor points out, make the poverty experts any less complicit.  They had, after all, conducted the studies upon which the legislation was based, and the fact that their findings were couched in “scientific” and “objective” language did not make their production any less of a political act.  The “end of welfare,” arguably a devastating blow to the poor, especially poor children, was not so much produced in the absence of liberal social scientific research but because of it:  politicians found legitimacy for their actions in liberal poverty expertise (O!Connor 2001, 286; 291).      46 The legacy of scientism:  The Coalition for Evidence-Based Policy Belief in the ability of scientifically rigorous evaluation studies to identify “highly-effective” social interventions—elaborated in the War on Poverty and further developed and utilized in the decades since—forms the basis for the Coalition for Evidence-Based Policy!s (CEBP) 2009 call (as stated on its Web site) for an Executive Order to “institutionalize evidence-based approaches in [U.S.] federal social programs.”  The Coalition!s board of advisors contains some familiar names:  David Ellwood, Isabel Sawhill, and Robert Solow, among others.  Ellwood!s role in welfare reform has been discussed above.  Sawhill co-directed a major Urban Institute initiative from 1981-82 to assess Reagan!s economic, social welfare, and new federalism policies.  The project lasted through the rest of the decade, published more than thirty volumes, and constituted the closest thing possible to a liberal response to the changes Reagan had instituted (O!Connor 2001, 145-46).  And economist and Nobel laureate Solow, using the latest econometric techniques, had constructed a detailed refutation of the structural unemployment concept—the idea that unemployment was a “structural” rather than an aggregate growth problem, and would not respond solely to high economic growth—for the Council of Economic Advisors! 1961 Economic Report (O!Connor 2001, 144-45). As elaborated on the organization!s Web site, the CEBP!s proposal for an Executive Order argues that many federal social programs set up to address important problems “often fall short by funding specific models/strategies ("interventions!) that are not effective.”  Mentioned in this category are K-12 educational curricula, job training projects, crime prevention efforts, and case-management assistance for low-income families.  (The proposal points out that the U.S. poverty rate now, at 12.5%, is slightly higher than in 1973.)  The specific proposal would charge the Office of Management and Budget (OMB) and other agencies to collaborate in advancing “two well-defined evidence-based reforms in appropriate program areas.”  The first of these is    47 “to grow the body of research-proven interventions by increasing the number of rigorous— preferably randomized—evaluations used to assess intervention impact (including cost).”  The second reform is “to spur the widespread use of existing research-proven interventions that produce sizeable sustained benefits (relative to cost), by incentivizing program grantees at the state/local level to implement them effectively” (emphasis added). As an example of a successful collaboration between the OMB and federal agencies, the proposal cites the OMB-Health and Human Services (HHS) welfare “demonstration waiver” policy in the 1980s and 90s. Under this policy, HHS waived certain provisions of federal law to allow states to test new welfare reforms, but only if the states agreed to evaluate their reforms in randomized controlled trials.  This policy resulted in over 20 large randomized controlled trials that produced valid, actionable knowledge about “what works” in moving people from welfare to work, and had a major impact on state and national welfare policy [i.e., the passage of the 1988 and 1996 welfare reform legislation] (emphasis added).   In an earlier paper published on their Web site in 2007, the Coalition elaborated on this example of “successful bipartisan welfare reform.”  From the Reagan through the Clinton Administrations, the federal government funded or facilitated more than 50 large-scale randomized controlled trials of welfare reform demonstration projects.  These studies built scientifically- valid, actionable knowledge about what works in moving people from welfare to work.  Of particular value, they showed conclusively that welfare reform programs that emphasized short-term job-search assistance and training, and encouraged participants to find work quickly, had larger effects on employment, earnings, and welfare dependence than programs that emphasized basic education.  The work- focused programs were also much less costly to operate.  This knowledge was a key to the bipartisan consensus behind the 1988 welfare reform act and helped shape the major 1996 welfare reform act including its strong work requirements.  These legislative changes led to dramatic changes in state and federal programs, and helped bring about major reductions in welfare rolls and gains in employment among low-income Americans (emphasis added).   The CEBP!s interpretation of the success of the demonstration projects and its positive spin on the 1988 and 1996 welfare reform legislation could not be more different from that of O!Connor.  “Poverty experts were initially triumphant,” O!Connor writes, “but in reality, the waiver    48 provision undermined the authority of expertise by making local officials and legislatures policy innovators.”  States experimented with stricter mandates on participants, such as immediate work for welfare, time limits, and benefit reductions or cutoffs for those who failed to “play by the rules.”  As welfare rolls began to swell in the early 1990s, state waiver requests turned to benefit cuts and then to the “work first” model of reform “despite widely available evidence that low-cost immediate placement programs did little to raise earnings and job security for the poor” (O!Connor 2001, 289).  This is the antithesis of the conclusions reached by the CEBP that programs that encouraged participants to find work quickly had positive effects on employment and earnings for the poor.  The reasons behind these very different interpretations of the effects of work first programs can be found in the CEBP!s emphasis on the cost effectiveness and welfare roll reducing results of the programs.  The CEBP appears to equate “effectiveness” (a term it uses quite frequently—and problematically—as a measure of whether or not an intervention should be supported and continued) with the goal of decreasing welfare “dependency.” O!Connor emphasizes the negative effects that the work requirements had on the poor themselves, especially single mothers who now had to scramble to find childcare while being required to take low-paying and low-skilled jobs that offered little in the way of training or opportunities for future advancement and higher wages.  For O!Connor, a program that is “effective” would be one that actually addresses ending poverty rather than ending welfare dependency among the “undeserving” poor.  Obviously, neither “effectiveness” nor “progress” is a value-neutral or objectively determined condition, but highly dependent on what one values as an outcome.  In British Columbia, similar disagreements have arisen over how to interpret welfare caseload reductions.  On the one hand, the provincial government, pointing to the falling numbers on the welfare rolls, has claimed that its work first strategy, modeled after similar strategies in the U.S., is responsible for people getting off of assistance and into work.  Among    49 the provisions of the 2002 Employment and Assistance Act, individuals deemed employable would be cut off all assistance after two years, unless officials permit exemptions.  Such individuals are required to be actively seeking employment while on assistance.  Opposition to the two-year time limit resulted in the addition of further exemptions.  But public and government debates continue to centre on the importance of employment, irrespective of its adequacy or relevance to a person!s situation, and the government!s emphasis, as in the U.S., continues to focus on ending dependency and fostering self-sufficiency (B. Wallace et al. 2006; Reitsma- Street and Wallace 2004).  Citing statistics that showed reductions in the number of welfare caseloads, provincial officials have hailed the work first program a success.  But critics, looking at the same data sets, have emphasized that the government, by focusing only on welfare exits as an explanation for the reductions, has failed to focus on diversion and reduced entry rates as being behind the numbers.  Mandatory waiting periods and other obstacles to seeking assistance have caused many who would qualify for assistance failing to do so.  “In other words, welfare caseloads [reductions are] less about people getting off assistance and more about fewer people accessing assistance”.  This example mirrors that of the interpretation of welfare reductions in the U.S.:  how the figures are interpreted depends very much on the desired outcome and on what would be politically palatable.  As the Canadian Centre for Policy Alternatives points out, the public would no doubt support moving long-term recipients out of welfare and into jobs as a significant policy achievement. But citizens would look unfavourably upon policies that appeared to put up obstacles to those in poverty from getting the help they need (B. Wallace et al. 2006, 2l).       50 Summary  What does the example of poverty knowledge in U.S. history have to tell us about the current enthusiasm for evidence-based policy?  First, it demonstrates the importance of an historical perspective for understanding that What matters in determining whether and how knowledge connects to policy is not only the classical enlightenment properties of rationality and verifiability; nor is it only the way knowledge is mobilized, packaged, and circulated; nor even whether the knowledge corresponds with (or effectively shatters) popularly held values and conventional wisdom.  All of these things have, indeed, proved important….Even more important…has been the power to establish the terms of debate—to contest, gain, and ultimately to exercise ideological hegemony over the boundaries of political discourse (O!Connor 2001,17-18 emphasis added).  The problem is not always that decision-makers are not listening to researchers and evidence, or are only listening selectively.  Sometimes policy makers are listening to a particular side of an academic debate, seizing upon concepts and terms still poorly defined and very much contested, and using them to further their own political agendas.  This happened with the concepts of dependency and the culture of poverty, and as will be discussed in Part Three, it continues with policies formed around concepts such as social exclusion, concentrated poverty, and social mix, among others.  Academic debates over essential concepts are common, particularly in the social sciences, but researchers need to be aware that such debates within disciplines can have consequences when a lack of consensus is used as a justification to reject some analyses in favour of others, or to discount evidence altogether.  The trumping of community action research by quantitative, cost/benefit, and controlled experiment approaches to the evaluation of antipoverty programs illustrates the danger pointed out by critics of positivist approaches to research and policy analysis and modern EBP: that knowledge generated through social learning, lived experience, and methods (such as participatory action research), designed to address both academic and community-defined    51 research objectives, is frequently discounted in favour of externally-defined, quantifiable evaluation criteria.  This continues to be a problem.  The preference for quantitative research methods and randomized controlled trials is clearly stated in the Coalition for Evidence-based Policy!s criteria for determining so-called “top tier” intervention strategies.  In a 2008 letter from the Senate Committee on Homeland Security and Governmental Affairs to the General Accounting Office (available on the CEBP Web site), the Committee states that the CEBP, and others that advocate an evidence-based approach to program evaluation, recognize that not all programs are amenable to “rigorous statistical evidence.” These interventions, the Committee concludes, are no “less worthy of support than those whose goals are more easy to quantify.” However, this seems disingenuous, given the clear emphasis on, and the CEBP!s obvious preference for, the latter.  Over-dependence on government funding for research, and the co-opting of their own research methods and data by a proliferation of influential conservative and libertarian think- tanks, clearly undermined the ability of the poverty research industry to provide alternative analyses of the causes of poverty and its prevention. Rather than questioning the underlying assumptions, researchers undertook studies of the consequences of welfare “dependency,” thereby lending credence to the administration!s emphasis on welfare roll reduction and punitive workfare.  Critics who have emphasized the potential for the independence and critical role of social science research to be undermined by an EBP approach would do well to use U.S. poverty knowledge as an example of the consequences when this happens.  In her discussion of the various ways in which research knowledge can be utilized, Carol Weiss posits what she calls the “enlightenment model,” in which social science research affects policy development not through a single study or a body of studies, but through “concepts and theoretical perspectives that social science research has engendered that permeate the policy- making process” (1979, 429).  Research diffuses through a variety of indirect and unguided    52 channels—the mass media, for example, and now, of course, especially through the Internet— and the social science understandings that gain currency can be “partial, inadequate, or wrong,” bad science as well as good science, “endarkenment” as much as enlightenment (Weiss 1979, 430).  (Current debate around the reality of global warming is a particularly illustrative example.) Ideas about the poor—the culture of poverty, welfare dependency, the concentration of poverty and its attendant pathologies—were taken from social science literature, popularized by various authors, and became part of the received wisdom on the topic of poverty, as O!Connor so clearly illustrates.  Policy-makers were persuaded by these arguments, and policies as well as government-funded research were based on them.  The poverty knowledge example also illustrates the historical depth and persistence of ways of characterizing and socially constructing categories of people, as well as the history and continuing use of methods of gathering information about them. In the case studies that follow, homelessness policy, arguably one of the most research-receptive areas of housing policy, demonstrates the pervasiveness of scientism, a renewed and spreading enthusiasm for cost/benefit analysis, and how historically constructed beliefs about the poor/homeless (and how best to help them) continue to underlie contemporary approaches to homelessness interventions.  Social mix, a driver of housing policy that seems curiously immune to research evidence, also illustrates how essential an historical perspective is to understanding why we continue to be uncomfortable with spatial concentrations of low-income people, and why social mix appeals to such a broad spectrum of political persuasions.     53 Part Three:  EBP and housing policy The EBP approach has received considerable attention over the past decade from researchers who have examined its applicability and usefulness to the formulation, reform, and evaluation of housing policies.  As in other sectors, housing researchers, while generally encouraged that policy-makers are at least claiming to value and incorporate research results into their decisions, are cautious—some would say realistic—about the applicability of the EBP approach to housing, given the “multifaceted problems in which housing is implicated” (Doherty 2, 177).  The majority of discussions of EBP, especially in relation to housing policy, are from the UK and Australia. The interest in EBP in Australia stems from developments in the UK and has produced an analysis of the linkages between research and policy in Australian housing (Jones and Seelig 2004) and a systematic literature review of EBP sources (O!Dwyer 2004) in relation to housing, utilizing the systematic reviewing protocol guidelines produced by the Cochrane and Campbell Collaborations.  The similarities between the UK and Australian structures of governance make cross-national comparisons especially appropriate (O!Dwyer 2004, 1). In her review of the EBP literature in relation to housing policy, O!Dwyer (2004) concludes that, despite the “motherhood” statements of its proponents, there is no real evidence, with the exception of health, that evidence-informed policy is better than policy not informed by evidence.  She concludes that the approach is at least as effective and beneficial as non-EBP-based policy making methods, and superior at best.  Some commentators (Doherty 2000, 179-180; Reid 2003, 20 [cited in O!Dwyer 2004, 112]) have referred to EBP as an ideology, one based in scientism and one for which there is remarkably little supporting evidence.    54  A theme that comes out in much of the discussion of EBP and housing policy is the question of whether or not there is something inherently different or unique about housing policy that makes it more susceptible to ideology and less so to evidence.  Maclennan and More (1999, 23) have said this about British housing policy:  [E]vidence has an inconstant impact on UK housing policy and practice. Research communication methods and channels are improving but the substance of “messages” is not.  With poor data, housing policy and practice remain inordinately reliant, respectively, on polemic and fashion. Concrete evidence is spread thin on the ground and the housing policy edifice has shaky foundations.  Similarly Doherty!s work on UK housing policy indicates that increased housing research output seems to have had little or no direct impact on policy making.  He attributes this phenomenon to a mismatch between the widening and increasingly complex identification of the nature of housing problems and the type of research being conducted (2000, 179).  The problem of political context  In the end, the debate over whether and how to address housing needs and homelessness is a political problem, and there is no scientific or objective way to arrive at an answer to a political problem. The nature of the problem [in Canada] is well understood…[T]he questions about serious and effective government action on current housing and urban problems is a question about political will” (Hulchanski 2006, 232, emphasis added).   Hulchanski!s observation about political context addresses the fundamental lack of political will, at the federal level in Canada, to address problems in what he calls the secondary part of the housing system, i.e., housing based on social need rather than market demand. If Hulchanski is correct, then housing policy is less about evidence than it is about political context.  The evidence is there, he is saying, in the form of homelessness statistics, housing [un]affordability data, and the growing income disparity between renters and owners, and there is also a great deal of evidence about what could be done to effectively deal with those issues.    55 What a government chooses to do—or not to do—with that evidence is a matter of political ideology and expediency:  political context represents a major complicating factor in the attempt to establish an evidence base to policy making. Wright et al. (2007) tackle the problem of political context and the threat it poses to an EBP approach in their study of Britain!s New Deal for Communities (NDC), the Labour government!s urban regeneration program.  The NDC includes policies aimed at housing, education, crime, employment, and health in “disadvantaged” communities suffering from “social exclusion.” In their analysis of NDC documents, the authors attempt to assess explicitly how policy makers required evidence to be used to inform local decision making.  They found that while evidence was used early on as a means of informing the implementation of the policy, it became increasingly less emphasized as time went on and the wider political environment within which the NDC was being delivered was more explicitly recognized.  “Issues such as the requirement to demonstrate progress, financial imperatives, the hierarchy of local governance and mainstreaming, and abstract ideals such as citizenship, participation and social inclusion, all acted to constrain the simple application of evidence to policy making” (2007, 266).  The authors conclude that regardless of the means by which evidence is generated, the political environment will always distort the evidence message.  Rather than dismiss the attempt to introduce evidence into policy making as a waste of time, though, Wright et al. suggest that there appears to be at least a growing awareness among policy makers of the need to include evidence when developing and implementing policy.  They recommend that researchers should consider a portfolio of different approaches to EBP, recognizing that no one approach is likely to provide the answers to all the policy makers! questions (2007, 266). Wright et al.!s study illustrates the role played by socially constructed concepts such as “social exclusion” and “disadvantaged communities” in influencing public policy.  A collection of papers edited by Jacobs, Kemeny, and Manzi (2004) discusses the role of social    56 constructionsm in housing research especially as policy makers embrace and use concepts, such as social exclusion, before their utility and value have been thoroughly explored (Marston 2004, 78). Of equal relevance is the work of Schneider and Ingram (1993) on the importance of examining the social construction of target populations in the study of public policy.  By this the authors mean “the cultural characterizations or popular images of the persons or groups whose behavior and well-being are affected by public policy” (1993, 334). Schneider and Ingram contend that the “social construction of target populations has a powerful influence on public officials and shapes both the policy agenda and the actual design of policy” (334).  Some people are defined as deserving and worthy of programs and benefits, while others are negatively defined as undeserving and may become the targets of coercive and punitive policies aimed at changing their behaviour.  This process is well-illustrated in the poverty-knowledge discussion, as the constituents of the deserving and undeserving poor categories were constructed and reconstructed over time, based both on academic research and on popular characterizations promulgated by journalists and politicians. Social constructions enter into re-election strategies as public officials anticipate the reaction not only of the target population itself, but also of the wider (voting) public to whether or not the target population should be the beneficiary, or the loser, for a particular policy proposal (335).  “Hence, a great deal of the political maneuvering in the establishment of policy agendas and in the design of policy pertains to the specification of the target populations and the type of image that can be created for them” (336).  The following case studies, taken from different parts of the housing continuum, illustrate many of the points raised in the preceding discussion.  Homelessness policy and social mix both are predicated on concepts and beliefs that have long histories.  These include characterizations of “the poor,” ideas about the effects of the social and built environments on the so-called pathological patterns exhibited by low-income families, and cost/benefit    57 approaches to the evaluation of policy and program effectiveness.  These examples show that approaches that are touted as, and thought to be, new and progressive are actually based in ideas that have very deep roots.  Case Study One:  Homelessness research and policy in the U.S. and Canada  As Brushett has commented—and his observation is equally applicable to Americans— “[i]n the eyes of most Canadians, homeless people and emergency shelters to house them are…recent problems” (2007, 376).  Kusmer (1987, 21) made a similar observation twenty years earlier when he pointed out that “[c]ontemporary discussions about the recent increase in homelessness [are taking] place, for the most part, in an historical vacuum.”  However, the belief in the recentness of homelessness is erroneous.  Before the emergency shelter came the almshouse, or poorhouse, and while most Canadians and Americans might associate the institution with Dickensian England, the almshouse was also a feature of North American history, and in the United States, the almshouse dominated considerations of public relief policy for over a century and a half, and survived as recently as 1945 (Rothman 1987, 11).  Historian David Rothman provides this reason for not forgetting the almshouse:  Why is it so important to recollect the almshouse, to make this exercise in memory?  Because in fact we cannot understand the contemporary crisis around poverty in general, and homelessness in particular, if we do not appreciate the dynamics in the rise and, more or less, fall of the almshouse.  To be sure, the attitudes and social forces that determined its fortune do not operate in pure form today.  Nevertheless, understanding the history of the almshouse clarifies much that is puzzling in the predicament of the poor and the homeless today, and points us toward some useful directions for policy to take (emphasis added).  Beginning in the first decades of the nineteenth century and continuing until just after WWII, the almshouse door marked the distinction between the worthy and unworthy poor. Those who were in the first category—widows, the elderly, abandoned or widowed women with children—were to receive “outdoor” relief, that is, town funds or charitable assistance would be    58 given to them at home.  The unworthy poor—“moral derelicts…dope addicts, prostitutes, bums, drunks…the dregs of society [who] happen to need the institution as shelter for the moment” as a New York commission report described them in 1930—were to receive “indoor” relief, that is, inside the door of the almshouse (Rothman 1987, 12).  Almshouses were advocated as being a more economical way to deal with the poor, for those members of the public who blamed the poor for their circumstances, and reformatory, for the more charitable-minded who believed that hard work and proper training had the ability to rehabilitate the otherwise able-bodied. At least, this was the original intent of the almshouses and their supporters in the early twentieth century.  In fact, it was not the able-bodied, morally corrupt, intemperate but generally vigorous inmates who inhabited the almshouses, but a diverse group of the poor and homeless, including the elderly, the very young, the mentally ill and physically disabled, orphans, and the dispossessed and disconnected of foreign birth (Rothman 1987, 13). Reformers began to distinguish subgroups among the poor, and in an effort to separate the deserving from the undeserving poor—and to remove the deleterious influence of the latter—states began to provide centralized and specialized facilities, first removing children from the almshouses to orphanages and later, to foster and adoptive homes, and then removing the physically handicapped, “the insane,” and the mentally “retarded” from the almshouse to hospitals, asylums, and institutions built especially for them.  While these reforms may have improved the care of the mentally and physically handicapped to some extent, they were also closely related to efforts to abolish publicly-funded outdoor relief for that small and irresponsible minority that simply refused to work. The Depression, and the variety of New Deal programs designed to relieve poverty, led to the eventual demise of the almshouse in the U.S., and many of its earlier functions were taken over by other more specialized institutions, from hospitals to nursing homes to mental institutions (Rothman 1987, 16). With the proliferation of anti-poverty programs, and benefits    59 and insurance designed to support individuals and families since the 1930s—some of which were reversed with the welfare reform legislation of the 1980s and 1990s—came, ironically, a return to earlier distinctions between the deserving and undeserving poor.  But now, those distinctions are based on the belief that individuals whose needs are not being served by the social welfare system are somehow to blame for their own plight.  They must be addicted to drugs and/or alcohol, unwilling to work, or morally degenerate. However, like their almshouse antecedents, contemporary emergency shelters have increasingly come to house the mentally ill, many of them there because of the deinstitutionalization process since the 1980s, undoing the historical separation of the asylum from the almshouse (Hopper 1987, 97).  From street to home In contemporary North American homelessness policy, there is a growing emphasis on taking actions to prevent and end homelessness through a “housing first” model that puts homeless persons, and in particular, chronically mentally ill homeless, directly into secure housing, rather than focusing on the shelter-to transitional housing-to permanent housing continuum. Those individuals are then provided with the supports needed to remain housed. The pioneering housing first project, New York City!s Pathways to Housing, began operations in the early 1990s focussing on the chronically homeless mentally ill.  The program now boasts an 85-90% success rate in keeping previously homeless individuals housed, and has been recognized by the U.S. Substance Abuse and Mental Health Services Administration as an evidence-based practice (Burt et al. 2004).  While the Pathways to Housing model was originally developed out of a belief that housing is a basic human right, it also turned out to be highly cost-effective. It is considerably cheaper to house someone who is chronically homeless and mentally ill in his/her own    60 apartment and to provide support services than it is to keep that individual on the street consuming inordinate resources of the health, criminal justice, and social welfare systems.  As the model spread to cities across the United States and Canada, the cost benefits of the approach have inspired numerous research reports documenting the savings to taxpayers of the housing first model.  These demonstrations of cost-effectiveness have helped to sell the model even to those who are skeptical about the provision of housing to individuals who abuse alcohol and/or drugs, or engage in other behaviours that make them “unworthy” of assistance.  Million-Dollar Murray, cost-benefit analysis, and homelessness   In the tradition of Ken Auletta and Dwight Macdonald, journalist Malcolm Gladwell helped to bring homelessness and cost-benefit analysis to public attention through his 2006 New Yorker article entitled “Million-Dollar Murray” with the subtitle, “Why problems like homelessness may be easier to solve than to manage.”  Gladwell used the case of Murray Barr, a chronically homeless resident of Reno, Nevada, to discuss research on homelessness that showed that a relatively small group of chronically homeless individuals is costing the healthcare, criminal justice, and social welfare systems an inordinately large amount of money.  Providing these individuals with housing and support, the evidence showed, was much more cost-effective than allowing them to remain homeless and thus burdening the taxpayer through their overuse of services.  Murray!s moniker came from an estimate that he probably ran up an annual medical bill as large as anyone in the state of Nevada, and that over the ten years he was on the streets, his total medical expenses might have amounted to as much as a million dollars. Gladwell was referring to the work of Dennis Culhane, who was the first researcher to discover in the early 1990s, by tracking shelter users, that the homeless were not a homogeneous group, though they had been treated as such. The majority of homeless, rather than being chronically unhoused, was temporarily or episodically homeless.  However, it was    61 the chronically homeless, especially those who suffered from mental illness, who cost the health, social welfare, and criminal justice systems the most. Research conducted by Culhane and his colleagues, as well as subsequent investigations, have influenced and changed not only how policy makers and the public perceive homelessness, but also how evaluation of the effectiveness of homelessness interventions is approached, and how homelessness programs can be made more accountable to consumers, funders, and the public.  In a paper published in a collection from the 1998 (U.S.) National Symposium on Homelessness Research, Culhane et al. (1999) discussed how various types of performance measurement could be used to improve the accountability of homeless programs to stakeholders. The authors made a number of research recommendations in their article including the development of instruments that could measure consumer outcomes, provider performance, and systems effectiveness and efficiency.  They emphasized the need for a closer relationship between providers and researchers so that provider-identified issues could become the focus of formal research projects.  The authors especially stressed the need for a means by which to track the outcomes of consumers longitudinally, in particular, the use of automated systems for the tracking of homelessness services.  While they emphasized the importance of the establishment of benchmarking measures for performance, Culhane et al. stressed that consumers, providers, funders, and researchers all need to be involved in developing a common understanding of what services and units of services mean in different settings in order to make meaningful comparisons.  The authors discussed cost-benefit analyses—studies that examine the economic benefits of providing services to homeless people as opposed to not providing those services—as a way of demonstrating accountability to the taxpaying public. In a follow-up paper, Culhane et al. (2008) assessed how U.S. homelessness research and program evaluation had developed since 1998.  What they found was that the majority of    62 research in the ten-year period had focussed on the costs of homelessness and the cost offsets associated with various interventions, and that the majority of this research focussed on the target group of the chronically homeless mentally ill.  Literature on cost, cost-effectiveness, and program outcomes for populations who are not chronically homeless or who do not have a severe mental illness had grown much less (though a recent study released in March 2010 examines costs associated with first-time homelessness for individuals and families, indicating the expansion of the approach to other homeless populations [Spellman et al. 2010]). The authors attribute the greater prevalence of research on homelessness among people with mental illness, in part, to the funding priorities of federal research sponsors.  They point out, however, that people without severe mental illness constitute the majority of people who experience chronic homelessness. Those with mental illness make up about thirty percent of the chronically homeless, and the chronically homeless, as a group, make up only about ten percent of the homeless population.  Thus the majority of U.S. research has been targeting a small subgroup within a small subgroup at the extreme end of the homeless distribution, and a substantial portion of that research is intended not so much for an academic audience, but for a policy audience (Culhane et al. 2008, 12-1-12-3). Cost-benefit studies, in particular, are playing an instrumental role in policy discussions in the U.S., especially within cities where they are undertaken. Many municipal governments and service providers are using the documentation of the high costs associated with a subset of homeless people—the chronically homeless mentally ill—as a means of demonstrating the economic impact of chronic homelessness on society and thus garnering political momentum around plans to address it.  The U.S. Interagency Council on Homelessness has encouraged communities to identify the most expensive persons, regardless of how representative they are of the local homeless population, because, as high service users, they are people who can be housed with significant reductions in costs. Indeed, cost/cost-effectiveness research indicates    63 that expensive interventions, like housing provision, are not likely to be cost-effective except for the most costly clients. United States federal housing policy has also been affected by research on the costs and cost-effectiveness of homeless assistance programs. Since 2000, Congress has required that thirty percent of federal housing dollars be spent on permanent housing, and of that, one third is to be set-aside for projects that serve a population that includes at least seventy percent chronically homeless persons. Other cost studies have been directly inspired by Gladwell!s New Yorker article (Culhane et al. 2008, 12-7; 12-15).  Million Dollar Murray, and how most effectively to provide services to those like him, has become part of the collective knowledge about homelessness.  In Canada, homelessness research has also focussed on the cost-effectiveness of investment in supportive housing and the benefits of a housing first approach.  Studies conducted in Vancouver, Calgary, Ottawa, Toronto, and Halifax have all documented the expensive strain on a number of systems (health, criminal justice, social services) and the savings to be gained by putting homeless individuals into housing as opposed to providing shelters and other services aimed at managing homelessness.  The Canadian Housing and Renewal Association (CHRA) in its Policy Position on Homelessness (available on the CHRA Web site), has come out in strong support of a national strategy to end homelessness as not only the morally correct response, but also the most cost effective. Similarly, the Mental Health Commission of Canada—created in 2007 by the federal government to develop the nation!s first national mental health strategy—has come out in support of the housing first model. It is funding research studies in five cities to learn more about effective ways of helping the homeless mentally ill (Kirby 2008). In the kind of randomized controlled study that EBP proponents favour, a total of 2,285 mentally ill homeless in the five cities have volunteered to participate in the studies, and of those, half will be given their own    64 housing along with support services, while the other half will receive services and temporary shelter.  Researchers hope to determine the effectiveness of stable housing, and what types of services would work best to reduce homelessness in Canada.  Cost-benefit analysis will form an important component of this research: according to one of the research directors, “[u]nstable living and being on the street is not good for the health of societies, because people end up being in desperate situations where they use a lot more of our resources.”  She went on to name some of those resources, such as emergency rooms, police services, and incarceration as the costly results of homelessness (Vancouver Sun, November 23, 2009, A10, emphasis added).  A later editorial piece in the Sun (December 4, 2009, A14) described the research project as “look[ing] at the comparative costs to society of providing people with housing or leaving them on the street.”  Re-creating the deserving and the undeserving homeless While advocates for the homeless, planners, academics, and social policy analysts in both the U.S. and Canada can only applaud the advances in our knowledge about homelessness represented in recent research, one of the unintended consequences of the work by Culhane and other researchers has been the relative neglect of the majority of the chronically homeless population—those with primary substance abuse problems, for example—because such people do not reach the level of cost that would spur major investments in new housing and service interventions.  Families and individuals who make up the majority of the non- chronically homeless—those who are episodically or transitionally homeless (often for short periods of time)—have received much less attention from researchers and policy makers even though they constitute the majority of the homeless population.    65 The target population for homelessness research and policy—the chronically homeless mentally ill—has been socially constructed by public officials and policy makers as deserving of interventions, to use Schneider and Ingram!s analysis (1993).  Providing housing and services to this population “sells” from a political perspective, but as Gladwell has observed, “[s]ocial benefits are supposed to have some kind of moral justification….Giving the homeless guy passed out on the sidewalk an apartment has a different rationale.  It!s simply about efficiency….There isn!t enough money to go around, and to try to help everyone a little bit…isn!t as cost-effective as helping a few people a lot.”  Power-law solutions [i.e., those based on providing the most services only to those at the extreme end of the homeless population] have little appeal to the right, because they involve special treatment for people who do not deserve special treatment [and thus the need to re-construct them as “deserving”]; and they have little appeal to the left, because their emphasis on efficiency over fairness suggests the cold number-crunching of Chicago-school cost-benefit analysis [and thus the need to define such an approach as a “moral” as well as a cost-effective solution].  It!s hard not to conclude, in the end, that the reason we treated the homeless as one hopeless undifferentiated group for so long is not simply that we didn!t know better.  It!s that we didn!t want to know better.  It was easier the old way (Gladwell 2006, 104; 106, emphasis added).  The “old way” involved soup kitchens and emergency shelters rather than the provision of housing to the chronically homeless.  Everyone got a little rather than one subgroup getting a lot.  In some respects, the research of Culhane and others has been enormously successful in influencing policy by pointing out the diversity among the homeless population and by emphasizing that doing nothing but providing band-aid measures was more costly than giving at least some people homes.  In other respects, though, the emphasis on cost-offset and cost effectiveness studies has relegated the value to society of reducing homelessness to a discussion of dollars and cents.  As Rosenheck (2000) notes, “arguments can be made that housing is a social necessity in an advanced society such as ours, and that we have a collective    66 responsibility for making sure that resources are distributed in such a way as to assure everyone has access to housing.  Such arguments deserve further consideration, irrespective of the cost-effectiveness of particular homeless assistance programs” (Culhane et al. 2008, 12-16, emphasis added).  This is essentially an argument for a societal morality.  In addition, the focus on homelessness, its treatment and prevention, has tended to come at the expense of greater attention to the housing affordability gap that is in many ways contributing to the problem.  Culhane and Metraux (2008, 111-112) liken this approach to reallocating (rather than merely re-arranging) the lifeboats on the Titanic: while a reformed homelessness assistance program may not solve the problem of housing unaffordability, it can prevent involuntary shelter stays and reduce the time people spend homeless.  In a similar vein, the CHRA, while welcoming the shift in Canadian homelessness policy from managing homelessness to a housing first focus, nonetheless takes the federal government to task for the lack of a national housing strategy, and for having made no commitment to sustainable and on- going funding to increase the supply of affordable housing (CHRA Web site, “Policy Position”).  This example of the effects of research evidence on homelessness/housing policy illustrates the complexity of the relationship between evidence and policy.  Certainly, the policy focussing on the chronically homeless mentally ill is evidence-based, but it is also selective and very much an individualized approach.  In its emphasis on a case management model, housing first is the contemporary equivalent of the switch in emphasis within poverty knowledge from structural causes to a focus on individual pathologies.  Policy makers and service providers could embrace the research results because they provided a target group that could gain sympathy from politicians and the public, and that constituted an identifiable group of real people, with real needs, that could be housed.  Homelessness was a consequence of their illness, not the result of personal inadequacies.  Something can be done for them and at a cost    67 that is considerably below that of merely supporting them while they remained homeless, and success can be measured by how long they remain housed.  They are the deserving poor.  In both the U.S. and Canada, housing first models and interventions targeted to the homeless mentally ill and addicted provide both moral cover for governments—they can be seen as doing something “good” for these unfortunate people—and an economically based rationale consistent with the neo-liberal emphasis in both countries on cost-effective social policy. It was a win-win for governments and policy makers—the research evidence successfully penetrated the ideological filter.  Homelessness, after all, especially in terms of the chronically or absolutely homeless living on the streets, is an in-your-face embarrassment for all levels of government.  Homelessness is also bad for business—part of the costs involved in its perpetuation include the losses incurred by the tourism and arts and culture industries, as tourists, conference attendees, and would-be concert-goers avoid downtown areas where the homeless panhandle and sleep rough (Patterson et al. 2008). (In Vancouver, host city for the 2010 Olympic and Paralympic Games, the Ministry of Housing and Social Development established a store-front office in the troubled Downtown Eastside to perform damage control in anticipation of the negative foreign press coverage of the area!s homelessness, drug addiction, and poverty.  The Ministry showcased its efforts in reducing homelessness and providing social housing as a way of mitigating potentially damaging and embarrassing media reports.) For human rights, housing, and anti-poverty advocates, however, the emphasis on cost- effectiveness produces a degree of discomfort.  Keeping someone homeless, regardless of whether or not it is more costly to do so, should not even be an option that is considered.  As Rosenheck said, housing seen as a basic human right should be provided to all regardless of cost. At the same time, however, most housing advocates recognize that, as morally repugnant as it may be, the cost-effectiveness argument is a powerful weapon in their arsenal when it comes to influencing policy makers and public opinion in the area of homelessness    68 interventions. As Culhane et al. pointed out, “cost-benefit analyses can provide powerful tools for homeless advocates in the policy arena…[q]uantitative arguments that a given intervention can save money relative to not investing in that intervention can go a long way to obtaining [needed] funding” (1999, 4-18).  This is an example of what Weiss (1979) has called the “political model” of research knowledge utilization.  Accountability and performance measurement Homelessness research has led not only to changes in how services are provided to the chronically homeless, it has also contributed to a greater attention to the use of evidence, whether collected by researchers or by service providers, in the evaluation of programs serving the homeless. However, studies of the costs of homelessness do no tell us much about the accountability or effectiveness of particular homeless assistance programs:  the question of what works? begs the corollary question of what doesn!t? Culhane et al. (1999) make a useful and important distinction among accountability to different groups of stakeholders, and what kinds of information each requires.  Consumers need data on the services they receive and whether or not those services meet their perceived needs. Funders need program-level performance data both to evaluate whether or not those programs are delivering the services they are funded to perform, and to compare providers based on standardized performance benchmarks. The public needs system-wide performance measures that demonstrate whether or not the system as a whole is improving the lives of homeless people, reducing the numbers of homeless, and to demonstrate if it is doing so in as efficient and cost-effective a manner as possible.  The authors argued for a broader, systems-wide view of homeless assistance programs that would examine the relationship of those programs to other, mainstream social welfare systems (i.e., those that did not specifically target the    69 homeless).  The authors encouraged the development of tracking systems, known as homeless services management information systems (HMIS), that could follow clients longitudinally and that could be used to assess program effectiveness on a more routine basis. To accomplish this, the authors urged closer collaboration between researchers and practitioners (Culhane et al. 1999, 4-7).  They cautioned, though, that “the performance of homeless providers is affected by significant externalities that are beyond the control of the homeless providers….This contingency makes any assessment of homeless providers! "performance! necessarily tentative and provisional” (Culhane et al. 1999, 4-12).  The British Columbia Auditor General!s report refers specifically to establishing accountability to the provincial legislature and the public through the establishment, by the MHSD, of performance measures; the report does not refer to accountability to the users of those programs.  The question of what works in the report begs the question for whom, and according to whose evaluation criteria.  To reiterate the point made by Innes (2002), the establishment of performance measurement criteria by the province runs the risk of missing the value of programs to those using them in favour of supposedly “objective” benchmarks.  It is probably fair to say that many service providers would be very uneasy with an evaluative system in which progress and effectiveness are determined and measured by people with MBAs and accounting degrees.  McAfee (1975) made the argument over three decades ago that, in order for any program evaluation to have an impact on policy, the process must assess not only aspects of those programs that reflect societal concerns—in the homelessness context, the cost- effectiveness of programs, for example—but as importantly, the end-users, the consumers of those programs! services, must be involved at all stages of the evaluation process. In her assessment of evaluations undertaken on a social housing program in the Greater Vancouver area, McAfee found that program recipients provided alternate perspectives on program    70 outcomes, goals, emerging problems, and directions for change.  Furthermore, policy and program reformulation proposals that were based on consumer input were received favorably by the recipients, agency personnel, and decision-makers (1975, 266-267).  The fact that researchers are still talking about the need for involving program recipients/consumers in the design of program evaluations indicates that there is still some degree of discomfort, at least in some sectors, with using consumer-provided input as evidence in policy formulation and in influencing changes in program design and delivery. This may be, in part, a reflection of the lower regard for qualitative data within the EBP approach, as pointed out by its critics.  However, the involvement of consumers, service providers, funders, and the public in the design and dissemination of research and evaluation results could also go a long way toward resolving the problem of the relationship between EBP and collaborative policy development.  Case Study Two:  Residential social mix  In a number of Western countries, social mix is a driving force behind urban and housing policy. Policy makers generally view social mix, in the form of mixed-income residential developments, as a solution to confront urban poverty. In the United States, for example, social mix has taken the form of the federally funded HOPE VI program to redevelop and revitalize the most egregious examples of failed public housing. It is combined with dispersal programs such as the national Moving to Opportunity for Fair Housing Demonstration Program that focus on relocating inner-city residents to lower-poverty communities outside the urban core  (Joseph 2006, 211; Fraser and Nelson 2008).  In Britain, social mixture of activities, uses, and tenures has become a guiding principle behind the Blair government!s programs for urban revitalization. While this is partly motivated by the desire to reduce social exclusion—hence the formation of    71 the Social Exclusion Unit within days of Blair!s election (Cole and Goodchild 2001, 355)—and build social capital, it also includes “a parallel discourse” that focuses on the eviction or exclusion of the undesirable (Blomley 2004, 88). As a powerful housing and urban development tool—evoking ideals of social equity, the value of diversity, and the perils of segregation by race and income—social mix is an idea with deep historical roots, stretching back to the nineteenth century, and a number of scholars Sarkissian (1976); Sarkissian, Forsyth and Heine (1990); Cole and Goodchild (2001) and August (2008) among others—have examined the origins, historical development, and ideological underpinnings of social mix.  Social mix is an extremely persuasive concept, and it is difficult to argue against it (cf. Blomley 2004, 90).  As Crump (2002, 581) points out, the concept also has appeal across the political spectrum: “[liberals] and conservatives alike argue that the social pathology of inner-city ghettos is largely the result of the spatial concentration of poverty.”  However, there remains considerable doubt among housing researchers about the efficacy of residential social mix and the wisdom of adopting it as a prime directive of housing policy.  Cheshire (2007, ix) has stated bluntly that “the ideal of "mixed communities! as a mechanism for achieving a measure of social equality” is “essentially a belief-based policy since there is scant clear-cut evidence that making communities more mixed makes the life chances of the poor any better.”  Indeed, a recent analysis of the U.S. Hope VI program, conducted by HUD researchers, found little support for the popular idea that improvements at the neighbourhood level, whether through redevelopment or relocation, lead to better outcomes for low-income individuals (Stoloff 2010).  Furthermore, there is a danger that, “researchers and policy makers have been trying to retrofit the analysis and evidence [for social mix] to support the "solution! fixed on by the pioneers of town planning in the nineteenth century,” that is, that by intentionally engineering the built and social environments, people!s lives can be improved (Cheshire 2007, 7).    72  Because the idea of social mix—or “balanced” communities—is so pervasive, influencing housing policy not only in North America, but also in the U.K., Australia, the Netherlands, and various other European countries, it has generated a large, diverse, and growing body of critical literature. In a review of a collection of these writings, Arthurson (2004,104-5) concludes that the available research demonstrates that there is an insufficient linking of the underlying assumptions made for social mix and the empirical evidence base.  Social mix is an example of a housing policy that is “based on the belief that…[it] is a "good thing! rather than on evidence that it works effectively” Maclennan and More (1999, 22). Arthurson notes that much of the contemporary policy debate on social mix “is occurring without recourse to the research findings” (2004, 104).  Indeed, Cole and Goodchild (2001, 359) refer to proponents! statements of the desirability of social mix as “incantations.” Arthurson suggests that an evidence-based policy and practice approach might be usefully applied in determining whether or not a more balanced social mix actually achieves the goal of positive outcomes for low-income tenants.  While she recognizes that EBP is by no means a panacea, Arthurson believes there is potential value in testing the approach in determining the worth of social mix policies, especially in the regeneration/revitalization of public housing projects in Australia. Arthurson suggests that the approach to the systematic synthesis and analysis of the evidence base advocated by The Campbell Collaboration would be a fruitful way to go about drawing some useful conclusions from the social mix literature.  The steps involved are: • Gathering published and unpublished studies • Identifying the aims of implementing social mix policies • Detailing how the problems social mix seeks to address are specified • Identifying the mechanisms and tactics used to effect social mix • Analyzing how the above impact residents • Describing the context of the studies in terms of historical time, place, and social organizations within which the mechanisms and tactics are activated • Assessing whether studies conducted in one context are applicable or replicable in another    73 • Commenting on the potential relevance of these findings to contemporary housing and urban policy   One of the first, and most obvious, challenges to anyone attempting a systematic review is the inconsistent use of “social mix” in the literature.  Not all authors use the term and among those who do, not all use it in the same way or to refer to the same kind of “mix.”  Among those who do use it, social mix can variously mean income mix; tenure mix; ethnic/racial mix (in conjunction with income mix); socio-economic mix; and the mixing of those who work with those who don!t—or various combinations of these.  Other terminology that refers to social mix without always using the term are:  areal or neighbourhood effects; (un)balanced communities; residualization; social exclusion; tenure or housing diversification; concentrated poverty (and its solution, deconcentration); mixed-tenure; and mixed-income. Poorly or undefined terminology is an inherent feature of housing policies in some countries. Several researchers (e.g., Jacobs and Manzi 1996; Maclennan and More 1999; Wood 2003; Marston 2004) have discussed policy makers! use of imprecise or empty terms such as “affordability,” euphemisms such as “social housing” (rather than “welfare housing”), terms whose meanings are contested (such as social capital), and socially-constructed policy metaphors, such as social exclusion, and point to the danger of developing research methodologies to measure terms! effectiveness before their utility, value, and meaning are fully explored.  Social mix policies are especially vulnerable to terminological fuzziness because they are built upon a foundation of concepts and assumptions that are contested and/or poorly defined.  These include: social exclusion; the concentration of poverty; residualization; disadvantaged communities; social capital; social cohesion; and even such basic concepts as community itself. As Fine (2001, 190) has said about social capital, social mix is “a sack of analytical potatoes.”    74 What is especially curious about the emphasis on social mix as a fix for social exclusion and the regeneration of housing estates is that it is being promulgated by the same governments—for example, the British Labour government—that are emphasizing the need for evidence-based as opposed to ideologically based policy.  But the evidence for the efficacy of social mix, as most researchers agree, is either unsupportive or inconclusive.  Whether or not social mix as a policy intervention “works” is very much in question, yet the support for it not only in Britain but also in a number of other countries persists.  Why?  How can a policy whose very foundations are shaky at best persist when it so obviously contradicts the whole premise of EBP?  Policy makers seem to simply take for granted that social mix is a good thing, so there is no reason to test its assumptions.  The rightness of social mix, it seems, has become common knowledge, and as such, a socially-constructed “fact.” Underlying social mix strategies is a set of assumptions and beliefs about the relationship between residential segregation and concentration of the poor and its connection to pathological behaviours and generally negative outcomes for residents. Supporters of social mix see breaking up this segregation, either by re-development (regeneration/revitalization) and the creation of new, mixed-income communities, or by dispersal (inclusionary zoning, housing vouchers) as a positive, healthy, and ethical thing to do.  These beliefs about concentrated poverty and its association with dysfunctional behaviour have deep historical roots.  Exploring those roots can tell us a great deal about the persistence of social mix, and why it seems relatively impervious to influence by evidence.  The historical roots of concentrated poverty:  slums, congestion, and “immoral contagion”  “The very condensing of [the poor] within a small space seems to stimulate their bad tendencies.” Charles Brace, founder of the Children!s Aid Society, 1855    75   In the previous discussion of poverty knowledge, the writings of sociologist William Julius Wilson were shown to have been influential in stimulating examination of the consequences of “neighbourhood effects” in perpetuating a “culture of poverty.”  Spatial concentration of poverty, along with the absence of employment opportunities and middle-class (i.e., employed) role models, Wilson argued, have resulted in generations of poor, inner-city youth failing to internalize the kinds of values that would allow them to participate fully in society, and furthermore, have perpetuated pathological behaviours that threaten the health of the entire city (Wilson 1987; and, inter alia, Arthurson 2002, 246; Crump 2002, 583; Fraser and Nelson 2008, 2128). Wilson, however, was a latecomer to the idea that the concentration of the poor—and the removal of the more affluent—results in moral and environmental degradation (cf. Crump 2002, 583).  In the mid-nineteenth century, American social reformers assumed that “once isolated from the moral influences of their social superiors, the poor would fall prey to the most depraved among them and thereby threaten the moral fabric of urban society as a whole” (Ward 1989, 14).  Youth were believed to be especially susceptible to the “immoral contagion” that progressed through the crowded tenements “as certain[ly] and as nearly as rapid[ly] as that of physical contagion” (Ward 1989,14).  A further concern was that this contagion, like communicable diseases and crime, could come to infect and affect the more affluent, as well. By the early twentieth century, “urban congestion” had become a main focus of attention of urban planners.  In 1907, a group of prominent New York urban reformers formed the Committee on the Congestion of Population (CCP).  The CCP identified low moral and intellectual standards of the mostly immigrant working classes, along with low wages, long working hours, and land speculation, as the causes of congestion.  Executive secretary Benjamin Marsh—whose 1909 book, An Introduction to City Planning, Democracy!s Challenge    76 to the American City was the first book-length American work on the topic—organized a huge exhibition at the Museum of Natural History to illustrate the evils of congestion.  Marsh advocated the transplantation of factories and housing to unsettled parts of the city, where land was still cheap.  He argued that this was the only way to improve living conditions for the working class and eliminate “the evil of congestion, with all the human suffering, physical deterioration, and moral danger which congestion promotes and connotes.”  Other charitable societies and muckrakers added weight to these arguments by documenting and publicizing the typhoid-ridden neighbourhoods, exhausted, exploited workers, overcrowded tenements, and generally appalling living conditions endured by industrial workers and their families, laying the blame on greedy landlords and despotic industrialists (Crawford 1995, 68; Marsh 1910, emphasis added). A New York Times article at the time decried the congestion of population in urban areas as “uneconomic,” since it led to an unbalanced distribution of population nationally, and resulted in thousands of deaths each year among the mostly unskilled immigrants crowded into the cities (November 28, 1910). In another article, the Times quoted from a meeting of the Board of Governors of the Real Estate Board of Brokers that emphasized the cost-savings of eliminating overcrowded living quarters, including reductions in “the cost of maintaining hospitals, the various asylums, and the jail, because of the better living of the greatest masses” (April 24, 1910). Nineteenth century concern over slums and congestion of population in cities illustrates the belief that overcrowding among certain classes and categories of people is physically and morally deleterious.  Reformers believed that dispersal of the population—through both the construction of industrial towns away from the urban core and the relocation of families to more salubrious living environments—would benefit those left behind (in the more affluent areas of    77 the city) and those who left.  We also see in this period the early expressions of the idea that economic (cost) benefits could be attained by improving the lot of the poor. In Canada, social mix has a similarly long history, and as in the United States, the idea has its roots in progressive and equality-minded principles.  August (2008) has traced this history from its origins in nineteenth century paternalism and utopianism that found its expression in factory towns, through the Garden Cities movement (see Sarkissian 1976) to a post-WWII enthusiasm in both Canada and the U.S. for social harmony and equality, and most importantly social stability, through the residential mixing of the social classes.  The 1960s and 1970s brought together Jane Jacobs! call for social mix as the foundation for the rebirth of modern cities with a growing dissatisfaction with Canadian and American urban renewal programs.   Reformers preferred the redevelopment of urban neighbourhoods as mixed-income areas to their destruction in the name of slum clearance and expressways. Income mix in housing, particularly social housing, entered Canadian federal housing programs as a way of not only contributing to the financial viability of such projects, but also of avoiding social problems “associated with projects which contained high concentrations of low-income households” (August 2008, 86-87). By the late 1970s and continuing to the present, Canada, the U.S., Britain, New Zealand, and Australia all experienced a neo-liberal shift from the idealism of the 1960s to an emphasis on the “free” market, individual responsibility, and limited government intervention.  This resulted in welfare state restructuring, social spending cutbacks (including housing), and an emphasis on public-private partnerships in the provision of social infrastructure.  In Canada, the federal government effectively ended any national housing policy in the early 1990s, and provinces and municipalities were forced to take on increasing responsibility for social programs including housing.  Within this context social mix re-emerged as the fix for concentrated poverty. Individual-level explanations for poverty merged with the image of higher-income individuals,    78 especially homeowners, as the epitome of respectability who can provide role models for the seemingly feckless residents of low-income neighbourhoods and housing estates (August 2008, 89; Doherty 2000, 182). While we no longer speak of “immoral contagion” the implication is clear:  the concentration of poverty produces social pathologies and the solution is the creation of more “balanced” socially-mixed neighbourhoods in which low-income renters can benefit from exposure to higher-income owners.  But unlike the nineteenth century reformers who, on the one hand focussed on the concentration of poor immigrants as the source of their communities! social ills, but nevertheless also blamed capitalist greed and exploitation for their plight, current rhetoric and policy take for granted the relationship between poverty and social pathology while failing to make explicit the connection between the causes of poverty and neo-liberal political, economic, and social policies.  As Blomley has noted, the problem with social mix is that it promises equality in the face of hierarchy (2004, 99).    79 Conclusion:  The lessons of history Researchers, planners, and housing advocates need to be knowledgeable not just about the political and ideological context of the government or bureaucracy they hope to influence, they also need to know the history and ideological context (e.g., social construction) of the underlying concepts of the policy. They need to realize that simply accumulating evidence will not necessarily bring about a change in policy if that evidence cannot penetrate deeply held beliefs with long histories. Similarly, policy analysts must also be aware of the historical roots of contemporary policies and their ideological underpinnings if they hope to understand their tenacity and resistance to change, and the varying political, economic, and social contexts in which policy is historically embedded.  Unarguably, the housing first model for addressing homelessness is preferable to the re-institutionalization model of the nineteenth and early twentieth centuries.  But, it is based upon a similar realization that the homeless are not all the same, nor are the causes of their homelessness, much as earlier social reformers recognized that the poor residents of the almshouses were not who society thought they were, and that the causes of poverty and homelessness were not solely the result of personal and moral failings. Housing first is an “outdoor” intervention, to use nineteenth century terminology—a policy that emphasizes not only giving people homes, but also supporting them there, once settled. It de-emphasizes the use of shelters, and it is an individualized response to the problem of homelessness.  The collection of data on the homeless—as emphasized in the British Columbia auditor general!s report, and reflected in the growing use of homeless management information systems to track the use of social and health services by homeless clients—is the present-day continuation of the strongly held belief in the value of a “scientific” approach to    80 homelessness and poverty interventions. Data (evidence) are viewed as essential for an informed and effective social policy, and the accumulation of data can become an end in itself by forming a corps of information upon which policy decisions can be based, and from which determinations of relative costs and benefits of specific interventions can be made. This faith in the power of statistics and quantitative data—which has led some critics, as discussed above, to label the approach as an ideology or a belief system rather than a method—is reflected in the evidence-based/informed policy approach.  The preference for quantitative over qualitative evidence is well-entrenched and has led to an almost complete inattention to the role of history in the analysis of what works (or worked) and the roots of the ideological assumptions that underlie many contemporary social policy approaches. Historical context should be an important part of an evidence-informed social policy.  The social work profession recognized this in the 1950s with the formation of the short-lived Social Welfare History Group, which sought to counteract the narrowly defined quantitative, ahistorical, and scientific approach embraced by the profession at the time.  The goal was to give social workers historical perspective that would provide a background for developing their ability to plan and to conceive social programs and policy-related legislation (Fisher 1999).  And an historical perspective is not necessarily incompatible with other kinds of analysis, such as cost-benefit. Is the policymaker always wise to pursue a cost-benefit analysis if a choice must be made?  Does the emergence of policy analysis as a discipline mean that historical examinations will become still less relevant?….In many instances, ideology and power have mattered far more than the newer forms of policy analysis would suggest….Perhaps we will need some form of cost-benefit analysis to generate solutions and some form of historical analysis to explain why the solutions are rarely adopted, or if adopted rarely work as planned (Rothman and Wheeler 1981, 11-12).  In regard to social mix, Sarkissian (1976) and Arthurson (2004) have written persuasively about the value of historical context to an understanding of social policy interventions.  Arthurson writes, “research needs to commence from an understanding of the    81 historical basis of social mix strategies.  Policies can be driven by different agendas and if the underlying beliefs and assumptions are not placed in their historical context, we may be doomed to repeat past errors” (2004, 105). In this, she echoes Wendy Sarkissian!s caution made over thirty years ago about the dangers of historical amnesia when it comes to policy analysis. It is important that planners should be aware of the long history of planning ideas [that] they frequently treat as modern phenomena. Perhaps a greater sense of humility and an historical perspective might deter [them] from adopting simplistic solutions to complicated problems, or making misstatements about current policies (1976, 243).  Innes has cautioned planners that, as much as they might want policy to be rational and scientific, nothing in the policy world is that simple.  Rational style and scientific evidence offer one perspective, and not always the perspective most useful to policy makers (2002, 102). We need to pay greater attention to the terminology we use and the loaded or poorly defined nature of much of it.  O!Connor mentions “dependency,” “hard core welfare users,” “recidivism” (in relation to welfare recipients), “intergenerational transmission,” and “working poor” as deserving of a moratorium on their use. We could add to those “cost-effective,” since “effective” itself is a loaded term; “social exclusion,” since its definition varies and it tends to deny that low-income communities have any “social capital;” and especially “the homeless,” with its one-dimensional characterization of a diverse group of individuals, and because it focuses attention not on the structural causes of poverty and inadequate housing, but on the supposed pathologies of their victims. Perhaps most difficult to implement, though, is O!Connor!s point that researchers need to acknowledge and embrace the inherently political nature of the knowledge they produce by opening up its usually buried interests and ideological assumptions to scrutiny and debate (294). When researchers undertake cost-benefit evaluations of programs and interventions, the outcomes of their research may serve to inform policies that end up doing more good than harm. But by engaging in those types of studies and speaking the language of cost/benefit, they    82 are participating in and supporting a methodological model that is itself based in the ideology of scientism, and that has its roots in evaluations of defense systems, not social programs. Society, and we as researchers, lose something when we give ourselves over to approaches that run the risk of diminishing our collective sense of responsibility to and for each other through the use of ill-defined jargon, confusing statistics, input/output models, and other technical apparatus that does more to obscure and mystify than it does to enlighten and inform. Finally, we need to continue to develop closer ties between academic researchers, service providers, and communities through the creation of joint research projects that privilege not only social science expertise, but also the lived experience of community residents and knowledge based in practice, and that address the information needs of service providers and their clients. Practitioners of deliberative/collaborative research and policy analysis approaches need to continue to push for the inclusion, in a meaningful way, of this kind of evidence as part of any approach to evidence-informed policy, and to engage in advocacy and public education to create a truly evidence-informed society.    83 References  Arthurson, Kathy.  2002.  Creating inclusive communities through balancing social mix: A critical relationship or tenuous link?  Urban Policy and Research 20(3): 245-261.  ———.  2004.  Social mix and disadvantaged communities:  Policy, practice, and the evidence base.  Urban Policy and Research 22(1): 101-106.  Auditor General of British Columbia. 2009. Homelessness: Clear Focus Needed.  Report 16, March 2009.  August, Martine.  2008.  Social mix and Canadian public housing redevelopment:  Experiences in Toronto. Canadian Journal of Urban Research 17(1): 82-100.  Bannerjee, Abhijit, and Esther Duflo.  2009.  The experimental approach to development economics.  Annual Review of Economics 1:  151-178.  Blomley, Nicholas.  2004.  Unsettling the city:  Urban land and the politics of property.  New York: Routledge.  Bremner, Robert.  1956.  “Scientific philanthropy,” 1873-93.  The Social Service Review 30(2): 168-173.  Brushett, Kevin.  2007. “Where will the people go”:  Toronto!s emergency housing program and the limits of Canadian social housing policy, 1944-1957.  Journal of Urban History 33: 375- 399.  Bryant, Toba. 2004. How knowledge and political ideology affects rental housing policy in Ontario, Canada: Application of a knowledge paradigms framework of policy change. Housing Studies 19(4): 635-651.  Burt, Martha, John Hedderson, Janine Zweig, Mary Jo Ortiz, Laudan Aron-Turnham, and Sabrina Johnson.  2004.  Strategies for reducing chronic street homelessness. Final report. U.S. Department of Housing and Urban Development, Office of Policy Development and Research, January 2004.  Chalmers, Iain. 2005. If evidence-informed policy works in practice, does it matter if it doesn!t work in theory?  Evidence and Policy 1(2): 227-242.  Cheshire, Paul.  2007.  Segregated neighbourhoods and mixed communities:  A critical analysis. The Joseph Rowntree Foundation.  Cole, Ian, and Barry Goodchild.  2001.  Social mix and the “balanced community” in British housing policy:  A tale of two epochs.  GeoJournal 51: 351-360.  Crawford, Margaret.  1995. Building the workingman!s paradise:  The design of American company towns.  London: Verso.    84  Crump, Jeff.  2002.  Deconcentration by demolition:  Public housing, poverty, and urban policy. Environment and Planning D:  Society and Space 20: 581-596.  Culhane, Dennis P., and Stephen Metraux. 2008. Rearranging the deck chairs or reallocating the lifeboats? Homelessness assistance and its alternatives.  Journal of the American Planning Association 74(1): 111-121.  Culhane, Dennis P., Wayne D. Parker, Barbara Poppe, Kennen S. Gross, and Ezra Sykes. 2008. Accountability, cost-effectiveness, and program performance: Progress since 1998. 2007 (U.S.) National Symposium on Homelessness Research.  Culhane, Dennis P., David Eldridge, Robert Rosenheck, and Carol Wilkins. 1999. Making homelessness programs accountable to consumers, funders and the public.  In Practical lessons:  The 1998 National Symposium on Homelessness Research, eds. L.B. Fosberg and D.L. Dennis, 4-1 to 4-23.  Washington, DC: U.S. Department of Housing and Urban Development and U.S. Department of Health and Human Services.  Doherty, Joe. 2000. Housing: linking theory and practice. In What works?  Evidence-based policy and practice in public services, eds. Huw T.O. Davies, Sandra M. Nutley, and Peter C. Smith, 167-186.  Bristol: The Policy Press.  Feuerstein, Marie-Therese. 1988.  Finding the methods to fit the people:  Training for participatory evaluation.  Community Development Journal 23(1): 16-25.  Fine, Ben. 2001. Social capital versus social theory: Political economy and social science at the turn of the millennium. London and New York: Routledge.  Fischer, Frank. 1990. Technocracy and the politics of expertise.  Newbury Park, CA: Sage.  ———. 1991.  American think tanks:  Policy elites and the politicization of expertise. Governance:  An International Journal of Policy and Administration 4(3):  332-353.  ———. 2003.  Beyond empiricism:  policy analysis as deliberative practice. In Deliberative policy analysis:  Understanding governance in the network society, eds. Maarten Hajer and Hendrik Wagenaar, 209-227.  Cambridge:  Cambridge University Press.  ———. 2009. Democracy and expertise:  Reorienting policy inquiry.  Oxford: Oxford University Press.  Fisher, Robert. 1999. “Speaking for the contribution of history:” Context and the origins of the Social Welfare History Group.  Social Service Review 73(2): 191-217.  Fontana, Andrea, and James H. Frey.  1994.  Interviewing:  The art of science. In Handbook of qualitative research, eds. N.K. Denzin and Y.S. Lincoln, 361-376.  Thousand Oaks, CA: Sage Publications.     85 Fraser, James, and Michael H. Nelson.  2008.  Can mixed-income housing ameliorate concentrated poverty?  The significance of a geographically informed sense of community. Geography Compass 2(6): 2127-2144.  Gladwell, Malcolm. 2006. Million-dollar Murray.  Why problems like homelessness may be easier to solve than to manage. New Yorker, February 13 and 20.  Greenwood, Davydd, William F. Whyte, and Ira Harkavay.  1993.  Participatory action research as process and as goal.  Human Relations 46:  175-192.  Guba, Egon, and Yvonna Lincoln.  1994. Competing paradigms in qualitative research. In Handbook of qualitative research, eds. N.K. Denzin and Y.S. Lincoln, 105-117.  Thousand Oaks, CA:  Sage Publications.  Hajdukowski-Ahmed, Maroussia.  1998.  Bakhtin without borders:  Participatory action research in the social sciences.  The South Atlantic Quarterly 97(3/4):  643-648.  Hajer, Maarten, and Hendrik Wagenaar, eds.  2003. Deliberative policy analysis:  Understanding governance in the network society. Cambridge:  Cambridge University Press.  Hammersley, Martyn. 2005. Is the evidence-based practice movement doing more good than harm? Reflections on Iain Chalmers! case for research-based policy making and practice. Evidence and Policy 1(1): 85-100.  Hopper, Kim. 1987. The public response to homelessness in New York City: The last hundred years.  In On being homeless:  Historical perspectives, ed. Rick Beard, 88-101. New York: Museum of the City of New York.  Hulchanski, J. David. 2004. What factors shape Canadian housing policy? The intergovernmental role in Canada!s housing system.  In Canada: the state of the federation 2004, municipal-federal-provincial relations in Canada, eds. Robert Young and Christian Leuprecht, 221-250.  Institute of Intergovernmental Relations, School of Policy Studies, Queen!s University.  Innes, Judith. 1998.  Information in communicative planning.  Journal of the American Planning Association 64(1):  52-63.  ———. 2002. Improving policy making with information. Planning Theory & Practice 3(1): 102- 104.  Jacobs, Keith, and Tony Manzi.  1996.  Discourse and policy change:  The significance of language for housing research.  Housing Studies 11(4): 543-560.  Jacobs, Keith, Jim Kemeny, and Tony Manzi, eds.  2004.  Social constructionism in housing research.  Aldershot:  Ashgate.  Jones, Andrew, and Tim Seelig.  2004.  Understanding and enhancing research policy linkages in Australian housing:  A discussion paper.  Australian Housing and Urban Research Institute, Queensland Research Centre.    86  Joseph, Mark L.  2006.  Is mixed-income development an antidote to urban poverty?  Housing Policy Debate 17(2): 209-234.  Kirby, Michael. 2008. The homeless and mental illness: Solving the challenge.  Presentation to the Collaboration for Change Forum, April 28, 2008, in Vancouver, BC. www.mentalhealthcommission.ca.  Krugman, Paul.  2007. Republicans and race. New York Times, November 19.  Kusmer, Kenneth L.  1987.  The underclass in historical perspective: Tramps and vagrants in urban American, 1870-1930. In On being homeless:  Historical perspectives, ed. Rick Beard, 20-31. New York: Museum of the City of New York.  Lennie, June.  1999.  Deconstructing gendered power relations in participatory planning: Towards an empowering feminist framework of participation and action.  Women!s Studies International Forum 22(1):  97-112.  Lepore, Jill.  2010.  Fixed.  The rise of marriage therapy, and other dreams of human betterment.  New Yorker, March 29.  MacLennan, Duncan, and Alison Moore. 1999. Evidence, what evidence? The foundations for housing policy. Public Money and Management January-March: 17-23.  Marsh, Benjamin C.  1910.  Causes of congestion of population.  Proceedings of the second national conference on city planning and the problems of congestion, Rochester, New York, May 21-24, 1910.  Boston:  National Conference on City Planning.  Marston, Greg.  2004.  Constructing the meaning of social exclusion as a policy metaphor.  In Social constructionism in housing research, eds. Keith Jacobs, Jim Kemeny, and Tony Manzi, 71-92.  Aldershot:  Ashgate.  Marston, Greg, and Rob Watts.  2003.  Tampering with the evidence:  A critical appraisal of evidence-based policy-making.  The Drawing Board:  An Australian Review of Public Affairs 3:143-163.  McAfee, Rosemary Ann. 1975.  Interactive evaluation: A user-oriented process to assist housing programme reformulation.  Ph.D. diss., School of Community and Regional Planning, University of British Columbia.  Nutley, Sandra, and Jeff Webb. 2000. Evidence and the policy process. In What works? Evidence-based policy and practice in public services, eds. Huw T.O. Davies, Sandra M. Nutley, and Peter C. Smith, 13-41. Bristol: The Policy Press.  Oakley, Ann.  2000.  Experiments in knowing: Gender and method in the social sciences. Oxford: Blackwell.  Oakley, A., D. Gough, S. Oliver, and J. Thomas.  2005.  The politics of evidence and methodology: lessons from the EPPI-Centre. Evidence and Policy 1: 5-31.    87  O!Connor, Alice. 2001. Poverty knowledge:  Social science, social policy, and the poor in Twentieth-century U.S. history.  Princeton:  Princeton University Press.  O!Dwyer, Lisel.  2004.  A critical review of evidence-based policy making.  Australian Housing and Urban Research Institute, Southern Research Centre Final Report No. 58.  Packwood, Angela.  2002.  Evidence-based policy:  Rhetoric and reality.  Social Policy and Society 1(3):267-272.  Parker, Ian.  2010.  The poverty lab.  Transforming development economics, one experiment at a time.  New Yorker, May 17.  Patterson, Michelle, Julian Somers, Karen McIntosh, Alan Shiell, and Charles Frankish.  2008. Housing and support for adults with severe addictions and/or mental illness in British Columbia.  Centre for Applied Research in Mental Health and Addiction, Faculty of Health Sciences, Simon Fraser University.  Pawson, R.  2002.  Evidence-based policy: In search of a method. Evaluation 8(2): 157-181.  Reid, F.  2003.  Evidence-based policy: Where is the evidence for it? University of Bristol School for Policy Studies policy paper.  Reitsma-Street, Marge, and Bruce Wallace. 2004. Resisting two-year limits on welfare in British Columbia. Canadian Review of Social Policy 53: 170-177.  Rosenheck, R.  2000.  Cost-effectiveness of services for mentally ill homeless people: The application of research to policy and practice. American Journal of Psychiatry 157: 1563- 1570.  Rossi, Peter .H.  1972.  Booby traps and pitfalls in the evaluation of social action programs. In Evaluating Action Programs, ed. Carol Weiss, 224-235. Boston: Allyn & Bacon.  Rothman, David J., and Stanton Wheeler. 1981. Social history and social policy. New York: Academic Press.  Rothman, David J.  1987.  The first shelters: The contemporary relevance of the almshouse. In On being homeless:  Historical perspectives, ed. Rick Beard, 10-19. New York: Museum of the City of New York.  Sarkissian, Wendy.  1976.  The idea of social mix in town planning:  An historical review.  Urban Studies 13: 231-246.  Sarkissian, Wendy, Ann Forsyth, and Warwick Heine.  1990.  Residential social mix: The debate continues.  Australian Planner, March 1990: 1-19.  Schneider, Anne, and Helen Ingram.  1993.  Social construction of target populations: Implications for politics and policy.  The American Political Science Review 87(2): 334-347.     88 Solesbury, William. 2002. The ascendancy of evidence.  Planning Theory & Practice 3(1): 90- 96.  Spellman, Brooke, Jill Khadduri, Brian Sokol, Josh Leopold, and Abt Associates. 2010.  Costs associated with first-time homelessness for individuals and families. Washington, D.C.:  U.S. Department of Housing and Urban Development, Office of Policy Development and Research, Office of Special Needs Assistance Programs.  Stoloff, Jennifer. 2010.  Guest editor!s introduction.  Cityscape:  A Journal of Policy Development and Research 12(1): 1-4.  Thacher, David.  2005.  Review of Maarten Hajer and Hendrik Wagenaar, eds. Deliberative policy analysis:  Understanding governance in the network society.  Journal of Planning Education and Research 24:  455-457.  Thomas, Hilary, Mark Petticrew, and David Morrison.  2001.  Health effects of housing improvement: systematic review of intervention studies. British Medical Journal 323: 187- 190.  Thomson, Hilary, and Mark Petticrew.  2004.  Assessing the health and social effects on residents following housing improvement:  A protocol for a systematic review of intervention studies.  MRC Social and Public Health Sciences Unit, Glasgow.  Tilley, N., and G. Laycock.  2000.  Joining up research, policy and practice about crime. Policy Studies 21(3): 213-227.  Wallace, Alison, Karen Croucher, Mark Bevan, Karen Jackson, Lisa O!Malley, and Deborah Quilgars.  2006.  Evidence for policy-making:  Some reflections on the application of systematic reviews to housing research.  Housing Studies 21(2): 2297-2314.  Wallace, Bruce, Seth Klein, and Marge Reitsma-Street.  2006.  Denied assistance: Closing the front door on welfare in B[ritish] C[olumbia]. Canadian Centre for Policy Alternatives—BC Office and the Vancouver Island Public Interest Research Group.  Ward, David.  1989.  Poverty, ethnicity, and the American city, 1840-1925.  Changing conceptions of the slum and the ghetto.  Madison:  University of Wisconsin Press.  Weiss, Carol. 1979. The many meanings of research utilization. Public Administration Review 39(5): 426-431.  ———. 1995.  The haphazard connection:  Social science and public policy.  International Journal of Education Research 23(2): 137-150.  ———. 2001. What kind of evidence in Evidence-Based policy?  Third International, Inter- disciplinary Evidence-Based Policies and Indicator Systems Conference, July 2001.  Wilensky, Harold.  1997.  Social science and the public agenda:  Reflections on the relation of knowledge to policy in the United States and abroad.  Journal of Health Politics, Policy and Law 22(5): 1241-1265.    89  Wilson, William Julius.  1987.  The truly disadvantaged:  The inner city, the underclass, and public policy.  Chicago:  University of Chicago Press.  Wood, Martin.  2003.  A balancing act? Tenure diversification in Australia and the UK. Urban Policy and Research 21(1): 45-56.  Wright, John S.F., Jayne Parry, and Jonathan Mathers.  2007.  “What to do about political context?”  Evidence synthesis, the New Deal for Communities and the possibilities for evidence-based policy.  Evidence and Policy 3(2): 253-269.  Yanow, Dvora.  2003.  Accessing local knowledge. In Deliberative policy analysis: Understanding governance in the network society, eds. Maarten Hajer and Hendrik Wagenaar, 228-246.  Cambridge:  Cambridge University Press.  Young, Ken, Deborah Ashby, Annette Boaz, and Lesley Grayson.  2002.  Social science and the evidence-based policy movement.  Social Policy & Society 1(3): 215-224. 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items