UBC Faculty Research and Publications

Autism and Lovaas treatment : a systematic review of effectiveness evidence Bassett, Ken, 1952-; Green, C. J. (Carolyn Joanne), 1956-; Kazanjian, Arminée, 1947- Jul 31, 2000

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
52383-Bassett_K_et_al_Autism.pdf [ 202.23kB ]
Metadata
JSON: 52383-1.0045257.json
JSON-LD: 52383-1.0045257-ld.json
RDF/XML (Pretty): 52383-1.0045257-rdf.xml
RDF/JSON: 52383-1.0045257-rdf.json
Turtle: 52383-1.0045257-turtle.txt
N-Triples: 52383-1.0045257-rdf-ntriples.txt
Original Record: 52383-1.0045257-source.json
Full Text
52383-1.0045257-fulltext.txt
Citation
52383-1.0045257.ris

Full Text

Centre for Health Servicesand Policy ResearchAutism and Lovaas treatment:A systematic review ofeffectiveness evidenceBCOHTA 00:1T            JULY 2000British Columbia Office of Health Technology AssessmentT H E  U N I V E R S I T Y  O F  B R I T I S H  C O L U M B I AAutism and Lovaas treatment:A systematic review ofeffectiveness evidenceKen Bassett  MD PhDSenior Medical ConsultantBC Office of Health Technology AssessmentCentre for Health Services & Policy ResearchUniversity of BCCarolyn J Green  BHSc (PT) MScResearch Co-ordinatorBC Office of Health Technology AssessmentCentre for Health Services & Policy ResearchUniversity of BCArminée Kazanjian  Dr SocPrincipal InvestigatorBC Office of Health Technology AssessmentCentre for Health Services & Policy ResearchUniversity of BCJULY  2000•BC Office of Health Technology AssessmentCentre for Health Services and Policy ResearchUniversity of British Columbia429 - 2194 Health Sciences MallVancouver  BCCanada   V6T 1Z3Tel: (604) 822-7049Fax: (604) 822-7975bcohta@chspr.ubc.cawww.chspr.ubc.caCanadian Cataloguing in Publication DataBassett, Kenneth, 1952-Autism and Lovaas treatment“BCOHTA 00:1T”Includes bibliographical referencesISBN 1-896256-16-31. Autistic children—Behavior modification.  2. Behavior therapy for children.I. Green, C.J. (Carolyn Joanne), 1956-  II. Kazanjian, Arminée, 1947-III.  British Columbia Office of Health Technology Assessment.  IV. Title.RJ506.A9B33 2000 618.92’898206 C00-910777-0© 2000 by British Columbia Office of Health Technology Assessment,The University of British ColumbiaPermission is granted to reproduce all or any portion of this report, providingacknowledgement is given to the authors.iiiForewordThe British Columbia Office of Health Technology Assessment (BCOHTA) wasestablished on December 1, 1990 by a grant to the University of British Columbia fromthe Province, to promote and encourage the use of assessment research in policy,planning and utilization decisions by government, health care executives, andpractitioners.  The Office does not participate in policy development for a requestingagency, its role is confined to appraisal of the scientific evidence.Assessments are performed in response to requests from the public sector such ashospitals, physicians, professional associations, health regions, government; privatesector groups such as manufacturers; and members of the general public.  One or moreof the following criteria are used to determine the priority of an assessment and the levelof analysis: (1) the number of users and potential change in quality of life; (2) theacquisition and operating costs to the health care system; (3) the potential to influenceprovider and consumer behaviour as a result of a review; and (4) the availability ofaccurate information and appropriate research skills.Health Technology Assessment projects are conducted by faculty and staff (includingmedical consultants) who are expert in systematic review methodology.  Electronicbibliographic databases and fugitive literature (that is, literature not indexed or distributedpublicly) are searched using predefined inclusion and exclusion criteria based on aspecific search strategy.  The critical appraisal of retrieved evidence includes theformulation of logical and defensible conclusions about the technology under study.Reports are reviewed internally, and then sent for external review to experts from avariety of academic or clinical disciplines.  Comments and suggestions are consideredbefore a final document is produced.  Distribution of reports is by request from the Officeor through inclusion on our mailing list, and reports are also available for publicdistribution.The strength of BCOHTA’s method of systematic review lies in the process of explicitlydetailing the methodology and criteria used to produce recommendations which arebased solely on the research evidence.  This transparent and reproducible assessmentprocess allows other investigators to review the evidence objectively for themselves.Arminée Kazanjian  Dr SocPrincipal Investigator, BCOHTACopies may be obtained from:BC Office of Health Technology AssessmentCentre for Health Services & Policy ResearchThe University of British Columbia Tel:(604) 822-7049429 - 2194 Health Sciences Mall Fax:(604) 822-7975Vancouver, BC   V6T 1Z3 http://www.chspr.ubc.cavTABLE OF CONTENTS Foreword ...................................................................................................... iii Acknowledgements ............................................................................................ viii EXECUTIVE SUMMARY ........................................................................................ ixPART I  •  REVIEW PROCESS INTRODUCTION .................................................................................................... 11 CONTEXT ........................................................................................................ 21.1 Autism ...................................................................................................... 21.2 Behavioural modification therapy .................................................................. 41.3 Scope of review ........................................................................................  51.4 Evaluation framework ................................................................................... 71.5 Research evidence versus human drama ........................................................ 82 METHODS ....................................................................................................... 92.1 Search protocol ........................................................................................  92.2 Critical appraisal of study methods .............................................................. 13PART II  •  SEARCH & CRITICAL APPRAISAL FINDINGS3 SEARCH FINDINGS ........................................................................................ 14 Table 1:  Summary of search findings ......................................................................... 15 Table 2:  Description of studies providing primary data .................................................... 164 CRITICAL APPRAISAL OF LOVAAS (1987) .......................................................... 174.1 Study appraisal ........................................................................................ 174.2 Critical appraisal findings ............................................................................. 204.3 Summary of critical appraisal of Lovaas .......................................................... 26vii5 CRITICAL APPRAISAL OF OTHER PRIMARY STUDIES ..................................... 275.1 Birnbauer and Leach (1993) ....................................................................... 275.2 Sheinkopf and Siegel (1998) ...................................................................... 295.3 Ozonoff and Cathcart (1998) ...................................................................... 315.4 Summary of critical appraisal of corroborative research ................................. 336 DISCUSSION ................................................................................................. 356.1 Lovaas effectiveness claim ......................................................................... 356.2 Failure of corroborative efforts .................................................................... 366.3 Does the evidence point a way forward? ........................................................ 376.4 Monitoring the state of effectiveness evidence ............................................... 386.5 Taking incremental, not programmatic steps .................................................. 407 CONCLUSIONS ............................................................................................. 41Bibliography .................................................................................................. 42References .................................................................................................... 55viiiAcknowledgementsThe observations and suggestions of the following reviewers have been extremely valuable in thepreparation of this report, and their contributions are most gratefully acknowledged.  Participationin the review process does not imply endorsement, however, and the British Columbia Office ofHealth Technology Assessment takes full responsibility for the views expressed herein.External ReviewRuth Croxford  MScResearch Co-ordinatorClinical Epidemiology UnitSunnybrook and Women's College Health Sciences CentreToronto, OntarioCanadaLinda C Eaves  PhD RPsychPsychologist, Autism Spectrum Resource TeamSunny Hill Health Centre for ChildrenChildren's and Women's Health CentreVancouver  BCCanadaAnton R Miller  MB ChB  FRCPCClinical Associate Professor,Department of Pediatrics, Faculty of Medicine UBCDivision of Developmental PediatricsCentre for Child Health and Health Evaluation ResearchBC Research Institute for Children's and Women's HealthVancouver  BCCanadaRuairidh Milne  MSc MB BS FFPHMScientific DirectorNational Co-ordinating Centre for Health Technology AssessmentUniversity of SouthamptonUnited KingdomKaren Tu  MD CCFP MScAssociate ScientistThe Institute for Clinical Evaluative SciencesNorth York, OntarioAssistant ProfessorDepartment of Family and Community MedicineUniversity of TorontoCanadaixEXECUTIVE SUMMARYThis systematic review examined whether early, intensive behavioural therapy for childrenwith autism results in normal functioning, or essentially a cure.  The scientific validity of thiscurative claim is central both to legal proceedings brought on behalf of several children inBritish Columbia against the Province seeking an intensive behavioural program; and tocost-benefit analyses and clinical guidelines used for planning autism treatment programs.The report concludes that, while many forms of intensive behavioural therapy clearly benefitchildren with autism, there is insufficient, scientifically-valid effectiveness evidence toestablish a causal relationship between a particular program of intensive, behaviouraltreatment, and the achievement of ‘normal functioning’.The following findings support this conclusion:1) The published literature on autism contains only one report, from a controlled clinicaltrial, in which the authors claim that their treatment normalized or cured children withautism. (Lovaas 1987, with McEachin et al 1993).  Although the study reported abenefit, it was small (19 children in the treatment group) and its findings of benefitcould have been achieved by assembling a high-functioning group of autistic children.The scientific community has been reluctant to accept the results of this study, notingthat while methodologically stronger than published reports of alternatecomprehensive therapies, the study is inadequate to establish the degree to which thisprogram of therapy results in children achieving ‘normal’ functioning, howeverdefined.2) The benefits in terms of overall functioning found by Lovaas 1987 (with McEachinet al) have not been corroborated by independent researchers.  Published controlledstudies involving this intensive behavioural treatment program do not report childrenachieving normal functioning as defined by Lovaas 1987 (with McEachin et al).Furthermore, uncontrolled studies, although a weaker form of evidence than controlledstudies because they do not account for the development process outside therapy,similarly do not support conclusions of ‘normalization’ through Lovaas therapy.It is recommended that randomized trials of alternative early intensive treatment programs areethical and feasible to advance research knowledge on the treatment of autism.  This researchis required before effectiveness claims can form the basis of public funding decisions regardingalternative program options.BC Office of Health Technology AssessmentAutism and Lovaas treatment1PART I  •  REVIEW PROCESSINTRODUCTIONThree British Columbia provincial government ministries, Health, Children & Families, andEducation, jointly requested the British Columbia Office of Health Technology Assessment(BCOHTA) to assess the effectiveness evidence regarding a program of intensive behaviouraltherapy known as Lovaas treatment, for pre-school children with autism.  The assessmentrequest reflects particular issues raised in legal proceedings filed against the Governmentof BC.The legal action, established as a judicial review in September 1999, alleges that theGovernment of BC is discriminating against children with autism by denying them their rightguaranteed under the Canadian Charter of Rights and Freedoms to public funding of a specifictreatment program.  In particular, the petitioners in the action claim that children with autismare being denied reasonable access to a program of behavioural therapy, in this report termed‘Lovaas treatment’, of sufficient intensity and early enough in their development to achieve,in some cases, normal adolescent and adult functioning, or at least a significant reduction inlater disability.In response to the request, BCOHTA conducted a systematic search for and critical appraisalof published scientific evidence, termed primary studies or data, regarding the impact of theLovaas form of intensive behavioural therapy on the overall outcome of children with autism.In addition, a systematic search was undertaken for published secondary analysis both criticaland supportive of this primary data.The report focuses on the effectiveness evidence of the Lovaas treatment program, not ofbehavioural therapy, also known as applied behavioural analysis.  Behaviour therapy isaccepted as a benefit to children with autism.  At issue is the large program, of whichbehavioural therapy is an essential component, that includes 40 hours per week of one-to-one therapy with specially-trained therapists, following a particular therapeutic manual.BCOHTA is established to undertake research of this nature, and to do so it receives anunencumbered annual block grant from the BC Ministry of Health.  The Office is not namedin the lawsuit, nor do any members of the Office have a financial or professional interest inthe provision of autism services.  The annual block grant will not be affected by the outcomeof the judicial review.  The presented findings are based solely on the research evidenceevaluated under BCOHTA’s method of systematic review, with open and fully reportedmethodology and criteria.BC Office of Health Technology AssessmentAutism and Lovaas treatment21.   CONTEXT1.1  AutismDefinitionAutism affects an individual's ability to socially interact with others, communicate, understandlanguage, and play.1  The World Health Organization 2 definition of autism encompasses thefollowing areas of developmental abnormality: 31) Social relatedness: abnormal social relationships and social developments2) Communication: failure to develop normal communication3) Imagination: interests and activities that are restricted and repetitive, rather than flexibleand imaginative.Autism is a lifelong neurological disability of unknown etiology.1  In terms of pathophysiology,it is generally accepted that:“[t]aken together, the available evidence in autism suggests that, although certainaspects of brain functioning are often spared in autism, the syndrome neverthelessinvolves widespread brain dysfunction at both the cortical and subcortical levels.The originating site of the brain injury has not been identified.” 4 (p137)Autism is diagnosed in approximately 15 of every 10,000 children.  With approximately40,000 live births in BC each year, this translates into about 60 new cases diagnosed per yearin the province, usually prior to school entry.Diagnosis and incidence estimatesRecent years have seen the development of consistent criteria for the diagnosis of autismspectrum disorders in both the DSM-IV (American Psychiatric Association, 1994)5 andICD-10.2  The National Institutes of Health (NIH) consensus group reports that based onvarious field trials, “the clinical diagnosis of autism remains one of the most reliable diagnosesin psychiatric or developmental research.” 4 (p123)These are recent developments, however, coming after the 1970s and 1980s when mostreported longitudinal outcome studies were conducted.  This is not to criticize earlierdiagnostic scales, but what has to be examined is the consistent application of any givendiagnostic scale during the course of an outcome study.A more recent problem has emerged in relation to estimates of the incidence of autism.It appears that, at least in the BC context, problems with the diagnosis of autism may notsimply be due to problems in the application of validated diagnostic criteria.  The possibilityarises that children with other pervasive development disorders may nonetheless be diagnosedwith autism in order to gain access to services linked to that diagnosis. 6  The extent of thisphenomenon or its influence on incidence estimates remains unknown.BC Office of Health Technology AssessmentAutism and Lovaas treatment3Long term outcome of children with autismIt is beyond the scope of the current project to conduct a systematic review of the scientificevidence regarding the long-term outcome of children with autism.  (For the purposes of thisreport, the long-term outcome of children with autism refers to their psychological, behavioural,and social development.)  It is nevertheless acknowledged that this must represent a significantelement in any framework of understanding.Clearly, however, the long-term outcome of children with autism is not uniform, nor is it wellknown.  The weaknesses of current knowledge are documented in a report by the US NationalInstitutes of Health, T e State of the Science in Autism, which notes that “Only a fewlongitudinal studies of children with autism have been conducted.” 4 (p141)Two review articles provide reasonable overviews of published longitudinal studies.7,8Howlin combines the findings from studies conducted in the two decades prior to 1970 withthe two decades since, and concludes that: “over the years, there has been improvement in thelevels of functioning attained by people with autism.” 7 (p55-6)  While the majority of individualsare ranked as ‘fair’ or ‘poor’, 10% - 20% of people in the latter years are in their own homesand in work.  Howlin qualifies the conclusions by noting that “direct comparisons betweenstudies are complicated, because of differences in methodology, in the subjects involved, andin data analysis.” 7 (p55-6)Nordin and Gilberg 8 provide a review article summarizing the findings from an unspecifiednumber of longitudinal studies of children diagnosed with autism, childhood psychosis orautistic-like conditions, some of which were reported as early as the 1960s.  Their summarycorroborates the findings of Howlin that:“These studies all yield remarkably consistent results … In all of the studies published,a poor or very poor outcome with regard to social adjustment was seen in 60 - 75% ofcases followed up to pre-adolescent or early adulthood.  A good outcome (with nearnormal or normal social life and acceptable functioning at work or school despitecertain difficulties in social relationships and oddities in behavior) was seen in 5 - 15%of cases.” 7 (p55-6)This conclusion is supported by the work of Kanner.  As reported by Howlin,7 (p55-6) “[i]n hisaccount of over 90 young adults who had first been diagnosed in childhood, Kanner noted that11% to 12% of the group had done relatively well, despite receiving little in the way ofspecialist intervention or support”.An appropriate conclusion from this longitudinal research is that, while most individuals withautism have poor to fair outcome, some individuals diagnosed with autism in early childhoodultimately do relatively well.  A conservative assumption based on these available data is thatany research regarding treatment effect bears the burden of proving that their study is notbiased by preferentially assigning children among treatment groups.It is worth dwelling on this point briefly for those unfamiliar with its significance toassessment of scientific validity.  Any treatment will appear effective or perhaps curative if itis applied to the (relatively small) subset of children who will do well regardless of specifictreatment.  The appearance of treatment effect will be further enhanced if, at the same time asBC Office of Health Technology AssessmentAutism and Lovaas treatment4children likely to benefit are assigned to receive treatment, children with poorer prognosesare assigned to a ‘matched’ control group.Moderate bias in favour of the treatment group could have a very significant impact onconclusions of treatment effectiveness, especially when the study group is small.  For example,biased assignment of 2 children out of twenty (10%) could have a 20% effect on assumptionof treatment benefit, if a child in the treatment group achieves normal functioning, while thechild in the control group remains significantly impaired.The only valid safeguard against this form of bias, known as selection bias, is randomassignment of children to treatment and control groups.  Random assignment of children totreatment and control groups has been recently shown by Jocelyn et al 9 to b  both feasibleand ethical in Canada, the most relevant context for this review.  This study, although only12 weeks in duration, is particularly relevant because it shows randomization is suitable forempirical validation of an early (24-72 months), intensive community-based integratedtreatment program.While their study is ground-breaking in this area of research, it was, as the authors note, tooshort to determine ultimate treatment effect.  The authors also point to the need to assesstiming, intensity and duration of their community-based program before drawing anyeffectiveness conclusions.91. 2  Behavioural modification therapyBehavioural modification therapy has its roots in the theory which holds that most behaviouris learned through interaction between an individual and the environment.  The therapy appliestechniques designed to affect this interaction and to improve behaviour.  The so-called ‘time-out’, for example, is designed to provide a child with a mild negative experience so as toreinforce rules against disallowed behaviour.Behavioural modification is to be distinguished from psychological or medical therapy.Psychological therapies aim to improve behaviour indirectly through developing individuals’understanding of themselves, their needs and desires.  In the classic Freudian psychoanalyticmodel, successful intensive therapy would lead to pervasive and long-lasting change.  Butthis therapeutic approach, in contrast to behavioural modification, is regarded as requiringintensive training by specialist therapists.  Behavioural therapy on the other hand is moreadapted for training parents and teachers to be part, and often the primary providers, of thetherapy itself.Medical therapy emphasizes biological determinants of human behaviour.  Some proponentsof this approach assert that most or all behaviour is dependent on physiological factors.The more generally-held and moderate view is that behavioural determinants include acombination of biological and learned elements.  Pharmacotherapy, widely used in the USand Canada for behavioural disorders 10-12 is a medical therapeutic approach, but almostexclusively considered symptomatic treatment.BC Office of Health Technology AssessmentAutism and Lovaas treatment5Applied Behavioural AnalysisApplied Behavioural Analysis (ABA) is a type of behavioural modification therapy.  It wasfirst defined in 1968 by Baer, Wolf, and Risley as “the process of applying sometimestentative principles of behavior to the improvement of specific behaviors and simultaneouslyevaluating whether or not any changes noted are indeed attributable to the process ofapplication.” 13  It emphasizes social (as opposed to clinical outcome) measures, focussingon changes in an individual’s social interactions, rather than a score against a self-conceptinventory.  ABA also emphasizes ongoing ‘analysis’ of the behavioural therapy so as toensure that the behavioural change is due to the therapy itself, and not to confounding factorssuch as influences from family or peers.The actual method for this ‘analysis’ is not clearly defined.  Rather, it is stated as a generalprinciple of ongoing critical analysis of effect.  Behavioural modification therapy thereforeseems to use what might be described as an ecological model.  Both individual behaviour andthe context in which it occurs are regarded as amenable to change, with an open possibility ofaltering either or both so as to effect improvement in behaviour.1.3  Scope of reviewUnder the terms of the request mentioned above, this assessment has a defined scopeof inquiry.At issue in this review are comprehensive behavioural treatment programs designed to alterthe outcome in autism, and improve the overall functioning of affected individuals.  Theseprograms address multiple symptoms, involve various professionals and parents, and requirethousands of hours of work over a period of several years.Issues not addressedThis systematic review does not consider the literature on focal treatments directed either atreducing specific behavioural problems associated with autism, such as sleep disturbances andescape behaviours; or at increasing behavioural successes such as social interaction with peersand symbolic play.  Following Rogers, among others, it is accepted that,“(t)he literature on effective focal treatment in autism is plentiful and published ina variety of journals, in the fields of developmental disabilities, applied behavioralanalysis, and discipline-specific journals … Behavioral treatment approaches areparticularly well represented in this body of literature and have been amplydemonstrated to be effective in reducing symptom frequency and severity as wellas in increasing the development of adaptive skills.” 14Matson et al 15 draw similar limited conclusions in favour of behavioural therapy for childrenwith autism.  The authors describe 271 published studies evaluating behavioural techniquesdirected at target behaviour, which are divided into categories of aberrant behaviour, socialskills, language, daily living skills, and academic skills.  (It may be noted that among thetechniques included is home-based, parent-mediated therapy - of particular interest here asone of a variety of parent-mediated therapies.)BC Office of Health Technology AssessmentAutism and Lovaas treatment6Nor does this review address several important issues related to the provision of medicaldiagnosis, treatment, or support services and educational opportunities for children andyouths with autism.For example, as mentioned above, there may be significant problems associated with labellingchildren specifically with a diagnosis of autism, as opposed to other pervasive developmentdisorders.  Children so labelled may have preferential service access not available to those withother diagnoses, thereby raising potential for an exaggerated incidence of autism diagnosis.Also unexamined are the challenges of sustaining standardized services across widely-dispersed geographic areas, such as exist in the province of British Columbia, other than toacknowledge, as the involvement of no less than three government ministries in developingan Autism Action Plan 16 suggests, that a wide range of specialists and special services arerequired in this field; and that the demands placed on these resources are likely to be increasedby the need to integrate the Autism Action Plan with government commitments to otherspecial needs children and their families.The focus on overall benefitThe Autism Action Plan which outlines the Government’s current and proposed commitmentsto children and youth with autism and their families, highlights “Early intervention andtreatment” as its first “Special Activity”.  It states:“Research supports what many parents have known instinctively: that earlyintervention for children between the ages of 0 and 5 can provide tremendousand lasting benefits.” 17The implication that early behavioural therapy can help to alleviate autistic symptoms in manyif not most children and provide significant relief for parents is a relatively modest claim andas such is not in doubt.  (Early intervention is here taken to mean starting therapy at ages 3 to 5years when autistic symptoms have become manifest, and which is earlier than wouldotherwise be possible within regular, publicly-funded educational services such askindergarten and main school.)  To reiterate, the literature supporting assertions thatbehavioural treatment alters many symptoms associated with autism is not considered in thisreport, except to note that it is extensive, with the weight of evidence favouring benefit.At issue in this report, however, is the stronger assertion, conceivably also to be inferred fromthe Autism Action statement, that scientific evidence supports behavioural therapy as a meansto alter beneficially the natural course of the condition, or all the symptoms of the condition,and as a result a substantial number of children will achieve ‘recovery’ or ‘cure’.At this point, the type of scientific evidence needed to support this assertion is brieflydiscussed and illustrated to assist those unfamiliar with standard evaluative frameworks.1.4  Evaluation frameworkIf the long-term outcome of a condition is well known, and predictably poor or fatal, then toprove any therapy capable of altering that outcome requires only a carefully documented caseseries observing treatment effect.BC Office of Health Technology AssessmentAutism and Lovaas treatment7A relatively simple example of such an approach is to be found in Tay-Sachs disease, a rareganglioside-storage disease, hitherto uniformly fatal in infancy.  Any treatment for thiscondition resulting in increased longevity or a moderate level of intellectual and socialfunctioning could be assumed to be due to treatment effect, and would therefore require nocomparison group to substantiate the scientific validity of an effectiveness claim.  The onlyissue of scientific validity is whether individuals receiving treatment were correctly diagnosedas having the relevant condition at the inception of the case series.As the long-term outcome of a condition becomes more complex and unpredictable, however,studies designed to assess that condition require increasing complexity if they are to sustainvalidity.  With autism for example, if some children ‘recover’, in the sense that they enterregular schools, and ultimately live independently in productive careers, proponents of aparticular therapy must prove an effectiveness claim has not been biased by selecting fortreatment those children who would have recovered with minimal, alternate, or less intensetherapy.To guard against such selection bias, study researchers should, theoretically, randomly selectpatients from a population under study.  However, because random selection is seldomfeasible for complex childhood developmental disorders, the best that researchers can achieveis to make explicit the extent to which their population is representative of a cross-section ofindividuals with the condition.An additional opportunity to minimize selection bias arises through random assignment ofchildren, however selected, to treatment or control groups.  If researchers have systematicallybiased their study through selection of children who will do well regardless of treatment, thiswill become apparent in an exceptionally good outcome found in the control as well as thetreatment group.  In almost no circumstances can strong conclusions be drawn in the absenceof random assignment.The Task Force on Promotion and Dissemination of Psychological Procedures examined abroad range of interventions for childhood disorders, and concluded as follows:“Given the advances in outcome methodology within the past 25 years … it seemedappropriate to establish an absolute set of criteria for determining adequate studydesign and empirical support.  For instance, because of threats to the internal validityof studies not using random assignment, random assignment in a group study wasconsidered an absolute criterion.” 18 (p141)A study design that includes randomized treatment and control groups, however, whilenecessary, is not sufficient to establish scientific validity.  Several additional conditions needto be met, as called for by the NIH:“Research is needed that uses robust experimental designs to evaluate and comparevarious approaches to treatment.  Methods are needed that (a) involve randomassignment to different treatment conditions; (b) use standard intervention protocolsthat capture a wide range of skills and symptoms, under both laboratory and ‘real life’situations; (c) make use of outside evaluators who are not invested in the outcome ofthe research; (d) assure high compliance with the defined treatment protocol to be surethat the intervention as designed is actually and consistently implemented; and (e) useBC Office of Health Technology AssessmentAutism and Lovaas treatment8longitudinal designs that evaluate treatment effects, both during the treatment itself,and at set points after the intervention has been accomplished.” 4 (p149)The importance of these additional conditions to the scientific validity of studies of childrenwith autism is presented in section 2.2 below, and the extent to which the individual studiesmeet them presented in the subsequent analysis, Part II.1.5  Research evidence versus human dramaThe research issues examined above should not be viewed as being only of academic interest,with no bearing on the realities of life for those affected.  Indeed, as has already been mentioned,the cost to individual children of poor research methods is not to be dismissed lightly.One difficulty confronting appraisal research, of considerable importance in the present case,is that the complexity of detailed health technology assessment reporting is unlikely to achievethe impact of mass-media accounts of dramatic ‘cures’.In the case of autism, the human drama projected by the popular press can be especiallycompelling.  If a specific therapy or program holding out hope of ‘cure’ is presented tofamilies facing daunting challenges, the impact on them is likely to be significant.  And withsufficiently high a public profile, the potential exists to influence funding-decisions andultimate program design, through policy-makers and program-providers, who after all arefamily people too.In these instances, however, it is important to recognize that what may seem a compelling‘association’ does not necessarily signify a ‘causation’.  Causation requires substantiationthrough an analysis of a group effect, not by individual cases.  This, in turn can only bederived from soundly-based research evidence.  It is therefore the strength of the evidencethat is at the heart of this review.The authors hold the view, that while argument for or against any particular position may beadvanced legitimately, if any intervention in a field which presents so many puzzling featuresis to be scientifically and impartially assessed, the integrity of the process must be paramount.The claims of individual need and the demands of public responsibility deserve no less.BC Office of Health Technology AssessmentAutism and Lovaas treatment92.   METHODS2.1  Search protocolI. RESEARCH  QUESTIONWhat is the effectiveness evidence that early, intensive behavioural treatmentprograms for pre-school children with autism result in improved overall outcomeversus alternative management strategies ?II. INCLUSION CRITERIA FOR PRIMARY DATA AND SECONDARY ANALYSISThe following selection criteria were used to identify the primary clinical study reports:1. Population of InterestHuman pediatric (pre-school) populations with autism; no exclusion due to presenceof co-morbidity.AND2. InterventionReports were included if interventions were described as early, applied behaviouralanalysis, behavioural therapy o  intensive, home-based program.All studies examining some form of overall autism treatment program were included.‘Early’ behavioural therapy was accepted as meaning initiation of therapy when thediagnosis is made and prior to age 5 when kindergarten services are available.AND3. Outcome measuresReports were included if the study measured overall function, including:· intellectual functioning;· language;· social interaction and play;· adaptive or self-care skills;· mal-adaptive behaviour;Studies were excluded that were limited to “training trials.”  Training trials are shortintensive efforts to alter a child’s communicative or social skills in a particulardomain.AND4. Types of studiesReports were included if the study design included a treatment and a control group.Individual case reports and case series were excluded.Articles providing critical appraisal of the primary study reports were also gathered.  Criticalappraisal is defined as: a systematic examination of the methodological quality of the reportsproviding primary study data.BC Office of Health Technology AssessmentAutism and Lovaas treatment10III. SEARCH STRATEGYStudy reports were identified by searching computerized bibliographic databases coveringtraditional medical literature: Current Contents, Embase, HealthStar and Medline.  A searchstrategy designed to identify primary analyses was combined with terms specific to autism.The following Medical Subject Headings (MeSH with first letter capitalized) and text words(keywords with no capitalization) were used in the Medline and HealthStar searches forconcepts which were combined.1. Autism terms: “Autistic Disorder;” “autis*;” “asperger*;” or “kanner*;” “lovaas*” or“lovaas-oi [as author];2. Co-authors of Lovaas or authors citing Lovaas: mceachin-j* or smith-t* or koegel-r* orrusso-d* or rincover-a* or newson-c* or wilhelm-h* or reynolds-b* or litrownik-a* ormann-r* or simmons-j* or leiken-s* or schaeffer-b* or perloff-b* or anderson-s* or avery-d* or dipietro-e* or edwards-g* or christian-w* or birnbrauer-j* or leach-d* or rogers-s* orlewis-h* or reis-k* or sheinkopf-s or harris-s or kanner-l* or mesibov-g* or mundy-p* orperry-r* or cohen-i* or decarlo-r* or boyd-r* or gresham-f* or macmillan-d* orwainwright-sharp-j* or bryson-s* or howlin-p* or goode-s*3. Intensive Behaviour Treatment terms: “Explode Behavior Therapy;” “Early Intervention(Education);” “early intervention;” “intense* or “intensive;” “discrete” or discreet” “triallearning;” “applied behavio* analysis;” “young autis*” “project” or study.”4. Controlled and Uncontrolled Trials terms: “Prospective Studies;” “Multivariate Analysis;”“Risk Factors;” “Odds Ratio;” “Evaluation Studies;” “Clinical Trials;” “Clinical Trials,Phase I;” “Clinical Trials, Phase II;” “Clinical Trials, Phase III;” “Clinical Trials, PhaseIV;” “Multicenter Studies;” “Comparative Study;”  “Sampling Studies;” “ProgramEvaluation;” “Case-control Studies” “Cohort Studies;” “Cohort Effect;” “LongitudinalStudies;” “Follow-up Studies;” “Prospective Studies;” “Retrospective Studies;” “Cross-sectional Studies;” “Survival Analysis;” “Research;” “Research Design;” “multivariateanalys*;” “risk factor*;” “odds ratio*;” “evaluation” “study or studies or trial*;”“comparison group design;” “historical cohort*;” “clinical or controlled” “study or studiesor trial*;” “multicent* or multi-cent*” “study or studies or trial*;” “comparative” “study orstudies or trial*;” “sampling” “study or studies or trial*;” “program* or programme*”“evaluat*;” “case control;” “cohort” “study or studies or analys* or trial* or effect*;”“longitudinal” “study or studies or analys* or trial*;” “follow-up or followup” “study orstudies or analys* or trial*;” “prospective” “study or studies or analys* or trial*;”“retrospective” “study or studies or analys*;” “concurrent” “study or studies or analys* ortrial*;” “incidence” “study or studies or analys* or trial*;” “cross-sectional” “study orstudies or analys* or trial*;” “case series;” “case” “study or studies or analys* or trial*;”“case design;” “time series;” “survival” “study or studies or analys*;” “uncontrol* or un-control*” “study or studies or analys* or trial*;” “non-control* or noncontrol*” “study orstudies or analys* or trial*;” “observational” “study or studies or trial*;” “researchdesign*;” and, as document type, “clinical trial;” “clinical trial, phase I;” “clinical trial,phase II;” “clinical trial, phase III;” “clinical trial, phase IV;” “controlled clinical trial;”“multicenter study.”BC Office of Health Technology AssessmentAutism and Lovaas treatment11The equivalent database-specific subject headings were used in the other databases searched.Text words only were used in Current Contents, as no subject headings are applied in this database.Databases searched were * Medline (1966-1999), HealthStar (1975-1999), Embase (1988-1999), CINAHL (1982-1999), Current Contents (1996-1999), and combined Science andSocial Sciences Citation Index (1989-1999).References of retrieved articles were reviewed to identify further relevant citations.The search results were reviewed independently by two reviewers.  Inclusion criteria givenabove were applied.  When differences arose as to whether an article was relevant to thereview, the disagreements were resolved by discussion.  All articles that appeared to meetthe criteria were requested in full text form.  The Bibliography at the end of this reportlists the articles obtained.IV. FUGITIVE LITERATURE SEARCHA search strategy similar to that adopted in the mainstream search was applied to thefugitive literature search, designed to identify primary analyses which combined termsspecific to autism.  Where possible, Medical Subject Headings (MeSH) and text wordswere used, but other subject headings were also applied to accommodate the indexingused in specific databases.  These subject headings were derived from Library of Congressor in-house indexing terms.Commercial databases:The following commercial databases were searched:1. Cochrane Library2. HSTAT (technology assessment guidelines)3. HSRProj (NLM)4. Dissertation Abstracts5. Article1st (OCLC)6. Papers1st (OCLC) – conferences and paper abstracts7. TRIP database (evidence-based medicine)8. Ebsco Academic Search9. Ebsco Canadian MAS10. Best Evidence11. CPG Infobase12. CRISP13. National Guideline Clearinghouse Database14. HTA DatabaseIn-house databases:1. In-house catalogueBC Office of Health Technology AssessmentAutism and Lovaas treatment12Web library catalogues:The following Web Library Catalogues were searched using Library of Congressor National Library of Medicine (MeSH) subject headings:1. UBC Library Catalogue2. BC Ministry of Health Library Catalogue3. Canadian Institute of Scientific and Technical Information (CISTI) Catalogue4. Belinda Database (Buckinghamshire Health Authority Library)5. WorldCat6. GAO Web Catalog7. LocatorPlus (NLM Catalog)8. Health Canada Library Catalog9. New Zealand Health Library Catalog10. Royal Society of Medicine Library CatalogueInternet peer-reviewed sites:1. UK Academic Web Directory2. UK Social Science Information Gateway3. OMNI (Organising Medical Networked Information)4. Medical Matrix5. Health Communications Network6. Global Health7. Health Index8. Medweb Public HealthInternet search engines:1. Northern Lights2. Google3. AltavistaDirectories:1. ECRI.  HealthCare Standards2. UHC Technology Assessment MonitorOrganizations contacted:1. Autism Society of BC2. Centre for the Study of Autism3. National Institutes of Health4. British Columbia.  Ministry of Social Services5. American Speech, Language and Hearing Association6. National Autistic Society7. Families for Early Autism Treatment8. Clark Institute of Psychiatry9. CMHS National Mental Health Services Knowledge Exchange Network10. United Kingdom.  Department of Health11. Autism Research Institute12. National Alliance for Autism Research13. National Institute of Neurological Disorders and Stroke14. National Institute on Deafness and Other Communication Disorders15. National Institute of Mental Health16. New York.  Department of HealthBC Office of Health Technology AssessmentAutism and Lovaas treatment132.2  Critical appraisal of study methodsBCOHTA critical appraisal methods require two independent researchers to assess studyvalidity using predetermined standard criteria.  The criteria applied are those highlightedby the NIH 4 (p148) and emphasized by other authors offering critical appraisal of the primaryeffectiveness data,19 namely that a study should:1. Compare various approaches to treatment2. Involve random assignment to different treatment conditions3. Use standard intervention protocols that capture a wide range of skillsand symptoms, under both laboratory and ‘real life’ situations4. Make use of outside evaluators who are not invested in the outcome ofthe research5. Assure high compliance with the defined treatment protocol to ensure thatthe intervention is actually and consistently implemented as designed6. Use longitudinal designs that evaluate treatment effects, both duringthe treatment itself, and at set points after the intervention has been accomplished.Detailed appraisal of each criterion follows accepted clinical epidemiology standards asoutlined by Sackett et al.20Study details were extracted independently by the two BCOHTA researchers.  Completeagreement was achieved regarding study design features.  Effectiveness data was alsoextracted independently by the two BCOHTA researchers, also with complete agreementregarding results.  Both BCOHTA researchers also read, discussed and agreed on theincorporation into the BCOHTA report of the critical appraisal findings of other authors.BC Office of Health Technology AssessmentAutism and Lovaas treatment14 PART II  •  SEARCH & CRITICAL APPRAISAL FINDINGS3.   SEARCH FINDINGSThe systematic search (electronic and reference lists) resulted in the identification ofapproximately 1200 abstracts and citations in total.  These were reviewed independentlyby two researchers.  Approximately 150 articles met the minimum inclusion criteria andwere retrieved.  The criteria were then re-applied to the full published reports to determineappropriateness for full critical appraisal.General features of the retrieved articles meeting the criteria are summarized in Table 1.Part A of the Table 1 lists research contributing primary data, and these studies are furtherdescribed in Table 2.No studies comparing alternative behavioural intervention programs, or randomized trialsof behavioural intervention programs were identified as meeting the inclusion criteria.Four controlled studies of treatment programs were identified that reported overall outcomefor children.21-25  The primary data from Lovaas (1987) 21 and McEachin et al (1993) 22 areappraised in section 4.  The two study reports will be reported and discussed together, sincethe report by McEachin et al is a long-term follow-up study based exclusively on the researchreported in 1987 by Lovaas.  The three further studies identified are appraised in section 5below.Studies contributing secondary analysis are listed in Part B of Table 1.  The first sectionincludes systematic reviews of effectiveness data in which the authors conducted criticalappraisal.  ‘Critical appraisal’ is defined broadly so as to include any reasonable evaluationof study design, procedures, analysis and outcome measures.  The requirement for criticalappraisal did, however, serve to exclude most review articles, which uncritically citeoriginal primary research findings.The second section of Part B includes all the published critiques and comments regardingthe primary study by Lovaas (1987) 21 and McEachin et al (1993).22  This latter category isconsidered essential because Lovaas (1987) 21 and McEachin et al (1993) 22 are the onlyauthors to date claiming to have evidence that their treatment program resulted in childrenattaining ‘normal functioning’.  The material serves both to determine the place of theseeffectiveness claims within the autism research community, and to obtain additional studydetails not originally included in the published trial reports.BC Office of Health Technology AssessmentAutism and Lovaas treatment15Table 1:  Summary of search findingsA.  PRIMARY DATATrials comparingalternative behaviouralintervention therapyRandomized trialsof behaviouralintervention therapyStudies of early (pre-school) intensivebehavioural therapy with a control groupNone None 1. Lovaas 1987; 21 (McEachin et al ‘93) 222. Birnbauer and Leach (1993) 233. Sheinkopf and Siegel (1998) 244. Ozonoff and Cathcart (1998) 25B.  SECONDARY DATASystematic reviews thatinclude critical appraisalof the primary dataCritical appraisal debates following Lovaas 1987 21 withMcEachin et al 1993 221. Rogers (1998) 262. Tregear et al (2000) 273. Howlin (1997) 74. Green (1996) 285. Smith (1998) 291. Schopler et al (1989); 30 response by Lovaas et al (1989) 312. Boyd (1998); 32  response Smith and Lovaas; 33response Boyd (1998) 343. Gresham and MacMillan (1997); 19  response Smith and Lovaas(1997); 35  response Gresham & MacMillan (1997) 364. Gresham and MacMillan (1998) 375. Schopler (1998); 386. Mesibov (1993); 39  Mundy (1993); 40  Kazdin; 41  Baer (1993); 42Fox (1993); 43  response by Smith et al (1993) 447. Connor (1998) 45BC Office of Health Technology AssessmentAutism and Lovaas treatment16Table 2: Description of studies providing primary dataREPORT DESIGN SUBJECTS INTERVENTION OUTCOMELovaas 1987 21(McEachin et al ‘93) 22Prospective:3-7 yrs.Assigned based onlocation / therapistsExperimental: n = 19Control 1: n = 19Control 2:  n = 19Age less than 40-46mths.Min. 40 hrs.Lovaas therapyBlindedExperimental:9/19 normal functioningControl: 0/19 normalfunctionsBirnbauer andLeach (1993) 23Prospective:2 yrs.Matched controlExperimental: n = 9Control:  n = 5Mean age 39 mths.Mean 29 hrs.Lovaas therapyNot blindedExperimental:no normal functioning4/9 achieved IQ ³ 89Sheinkopf andSiegel (1998) 24Retrospective:21 mths.Matched controlExperimental: n = 11Control:  n = 11Mean age 33 mths.Mean 27 hrs.Lovaas therapyIntake blindedExperimental:no normal functioningmean IQ 25 pts higherOzonoff andCathcart (1998) 25Prospective:10 -12 wks.Assignedfirst 11 children toExperimental; next11 children ControlExperimental: n = 11Control:  = 11Mean age 33 mths.10 home sessionsTEACCH therapyNot blindedExperimental:no normal functioningExperimental:improved 9 months ondevelopmental scaleBC Office of Health Technology AssessmentAutism and Lovaas treatment174.   CRITICAL APPRAISAL OF LOVAAS (1987)The principal citation identified in the search, and the study which underpins the claimsmade for the Lovaas treatment, is the 1987 research report by Lovaas (1987),21 togetherwith the long-term follow-up reported by McEachin et al (1993).22  This section providesa detailed account of this study and its critical appraisal using the criteria listed in 2.2 above.The remaining reports identified as offering primary data from controlled studies areexamined in section 5.4.1  Lovaas (1987), McEachin et al (1993) study appraisalStudy designThe description presented here includes details found both in the Lovaas (1987) 21 andMcEachin et al (1993) 22 study reports as well as material presented earlier as part of otherprogram accounts, and subsequently, in response to questions and criticism.This study was a prospective, non-randomized group study, comparing an active treatmentgroup to two control groups: one selected from children referred from the same source asthe treatment group and one group selected from the same source, but not referred to theLovaas treatment centre.Assessment at enrollment was conducted by psychology graduate students, ‘independent’from the investigators.  Assessment at enrollment was not described as blinded; that is,acceptance or rejection for active treatment could be known by parents.  In the case of5 children, assessment was conducted in conjunction with the children’s parents.Assessment at the initial outcome stage, school entry, in contrast to enrollment, wasconducted by researchers blinded to treatment allocation.Final outcome of school placement, IQ, and social functioning was conducted by researchstaff not blinded to treatment allocation; the 9 children who achieved the best outcome,termed ‘normally functioning’, were also reassessed using independent, blinded outcomeassessors.Inclusion criteriaAs stated in the original trial report: “Subjects were enrolled if they met three criteria:(a) independent diagnosis of autism from a medical doctor or licensed PhD psychologist;(b) chronological age (CA) less than 40 months if mute and less than 46 months if echolalic [*]; and(c) prorated mental age [**] (PMA) of 11 months or more at a chronological age (CA) of 30 months.The last criterion excluded 15% of the referrals.” 21 (p4)                                                          * echolalia means automatic repetition of what is said** prorated mental age is defined by Gresham and MacMillan 19 (p189) as “a psychometric scaling procedure.The PMA adjusts for variations in mental age scores as a function of the child’s chronological age at theBC Office of Health Technology AssessmentAutism and Lovaas treatment18Almost all children had clinical diagnoses of autism, made by an independent agency prior to contactwith the project.Group allocation method“… subjects were assigned to the experimental group unless there was an insufficientnumber of staff members available to render treatment (an assessment made prior to contactwith the family).” 21 (p4)“The assignment to groups was made on the basis of staff availability.  At the beginningof each academic quarter, treatment teams were formed.  The clinic director and staffmembers then determined whether any opening existed for intensive treatment.  If so, thenext referral received would enter the experimental group; otherwise the subject enteredthe control group.” 22 (p361)McEachin later adds detail that: “the first referrals for the study were all assigned to theexperimental group due to the fact that referrals came slowly (7 in the first 3.5 years) andtherapists were available to treat all of them.” 22 (p362)Population selected· An experimental treatment group of 21 children diagnosed with autism (2 lost to follow-up after 6 months).· Population selected over a 15-year period.· First control group consisted of 19 children referred for therapy, but for whom there werenot sufficient therapists available for a full intervention program; received 10 hours orless of 1:1 treatment.  Two children assigned to this group because they lived more thana one-hour drive from the treatment centre.· The second control group consisted of 21 children selected through a chart review fromthe same referral source, and who did not receive any of the intervention program.Study groups were found not to have any statistically significant differences in the followingdimensions:· chronological age at first diagnosis· chronological age at onset of treatment· prorated mental age· sum pathology (total of 8 measures of severity derived from parent interviews)· abnormal speech· self-stimulatory behaviour· appropriate toy play· any recognizable wordsThe only significant difference reported was a difference in the chronological age atonset of treatment.  Control group 1 subjects were 6 months older on the average thanexperimental subjects (mean chronological age of 35 months vs. 41 months, respectively).                                                                                                                                                                            time of test administration.  The PMA assumes a chronological age (CA) of 30 months, irrespective of thechild’s actual chronological age, and is calculated according to the formula: PMA = MA/CA x 30.”BC Office of Health Technology AssessmentAutism and Lovaas treatment19Therapeutic interventionThe intervention was reported as 40 or more hours per week of 1:1 behavioural therapy for2 or more years delivered largely by university students.  A succinct summary is providedby Rogers:“The therapy consisted of operant teaching techniques, mostly reinforcement but alsosome punishment techniques, used to teach a wide range of social, language, cognitive,and self-care skills, as well as to reduce inappropriate behaviors.  The initial phase oftreatment was delivered largely in the homes, with parents also trained and carryingout the treatment.  As the children progressed, behavioral teaching was delivered incommunity settings and typical preschools as well, with considerable focus ongeneralization and appropriate social behavior.  The treatment protocol was writtenup in a manual,[46] and instructional videotapes were also developed.  The studentsdelivering treatment were trained didactically in the principles of applied behavioranalysis in college courses and then trained to deliver treatment to the children throughadvanced trainers.  There was tight control of the treatment delivery by those who haddeveloped the treatment program.” 26 (p172)Lovaas adds that therapists require extensive theoretical and practical experience, as well asnoting that the “treatment effects could not be replicated without (contingent adversives)”.21 (p8)Outcome measuresInitial outcome measurements were conducted between the ages of 6 and 7, when a childwould normally have completed first grade.  IQ was measured using seven different IQ tests.Subjects were also scored as to their extent of normal functioning.  Subjects received a ‘3’for normal functioning if they received an IQ score in the normal range, completed firstgrade in a normal class in a school for normal children, and were advanced to the secondgrade by the teacher.Later follow-up of children in the experimental group and control group 1 (McEachin et al1993) 22 consisted of:· school placement ascertained from parents;· intelligence test (differing between children with and without verbal ability);· the Vineland adaptive Behavior Scales (administered to parents to assess child’s abilityto cope effectively with the everyday environment);· the Personality Inventory for Children (administered to parents to assess child’spsychological state and presence of disturbances).Findingsi)  At school entry, age 6-7 (Lovaas 1987) 21Clinically and statistically significant difference between groups on both overall outcomevariables:· IQ = 25-30 point increase favouring treatment versus either control groups;· functioning well in typical first grade classrooms without any special support:experimental treatment group (47%) versus controls (2%).BC Office of Health Technology AssessmentAutism and Lovaas treatment20Note that nearly half of the experimental group were termed “recovered”, having IQ scoresin the normal range (mean = 107; range 94-120) and passing through first grade in a regularclassroom.  School personnel described these children as “indistinguishable” from theirpeers.  However, no data was provided to substantiate this latter claim.ii)  Late childhood/early adolescence (McEachin et al 1993) 22· IQ = 30 point increase favouring experimental treatment (mean 84.5); control group 1(mean 54.9)· education in regular placements: experimental treatment 47%, controls (0%)· adaptive behaviour scores favour the experimental treatment group.McEachin et al conclude that the best outcome group of children sustained their beneficialoverall outcome.4.2  Critical appraisal findings of Lovaas (1987), McEachin et al (1993)For consistency with the published literature in this area, the critical appraisal issues will bepresented as answers to the NIH criteria 4 (p148) set out in section 2.2.  The discussion includesthe perspective found in the published literature (systematic reviews and debates) as well asthat of the BCOHTA reviewers.1) The study should compare various approaches to treatmentThe Lovaas study does not compare two different therapies, but only compares two levelsof intensity in a single form of behavioural intervention.  There were insufficient detailsprovided from the second control group to determine what therapies were applied.  TheBCOHTA researchers noted that the 10 hours of 1:1 therapy was not found to result in anychildren achieving normal functioning.2) The study should involve random assignment to different treatment conditionsGroup assignment was not randomized.  Lovaas et al 31 s er that the only criterion forassignment was availability of therapists and facilities.  The research team determined ifsufficient resources were available, and if so, then the next referred case was assigned tothe experimental treatment group.Baer 42 (p373) offers the only independent support for the group assignment method used inthe Lovaas study.  He states his conviction that the Lovaas method of group assignment, ifbased solely on predetermined resource availability, was essentially equivalent to randomassignment.  The researchers would have no input to group assignment and bias could onlyresult from referring physicians.He concludes that, because the children were so young and autism is so unpredictable, thatreferring physicians could not be “sensitive enough” to bias the treatment group to containhigher-functioning children with autism.  However, support for unbiased group assignmentis negated by Lovaas and his colleagues who admit that group assignment was also based onfamily factors, geographic location and stage of research.All other authors providing critical appraisal of the Lovaas study raise serious concernsregarding drawing effectiveness conclusions, in the absence of randomization.  Rogers raisesBC Office of Health Technology AssessmentAutism and Lovaas treatment21the most general and arguably most important point regarding randomization in studies withsmall sample sizes.  She notes: “Non-randomization is particularly problematic in studies withsmall sample sizes which are inherently at greater risk of group inequivalences that cannot bedetected using the usual statistical measures.” 26 (p172)Schopler et al 30 provide the most aggressive challenge to Lovaas (1987) in terms of thetiming and method of group selection.  They ask whether this study should be termed anexperimental study at all, or should more correctly be termed a ‘pos  hoc analysis’, that is,an incidental finding in a trial designed to answer a different question.Schopler et al 30 point to an earlier publication by Lovaas 47 which states that patientassignment was based on differences in the abilities of the family to provide the treatment.Families with restrictions such as divorce, personal problems, or full-time employmentcommitments were assigned alternate treatment placement of children in facilities.  Theimplication is that the families may have differed between the treatment and the controlgroups, contrary to what is claimed by Lovaas.  If so, the differences in findings may equallybe the result of differences in families rather than in the amount of treatment applied.In response to these criticisms, Lovaas et al asserted 31 (p166) that the only criterion forassignment was availability of therapists and facilities.  They explained that low-incomeand working families accepted the intervention because therapists were in their homesduring most of the day providing support and care for their children.However, despite this response by Lovaas, the BCOHTA researchers consider Schopler’spoint to be significant.  Questions remain about how the study was actually conducted atvarious stages:· Did the researchers follow a strict research protocol, established prior to enrollment ofthe first child, which included details of group assignment under various circumstances?In other words, how did the researchers deal with multiple more or less simultaneousreferrals?· How was one child selected for a single available experimental therapy placement?· Similarly, how did the researchers deal with subsequent group assignment if all 7 of theinitial referrals over the first 3.5 years were entered in the treatment group?· Were later stages of the research project influenced by earlier stages?  Several of theinitial seven subjects assigned to treatment would have entered school before subsequentstudents were enrolled.  An excess number of students were evidently assigned to thecontrol group owing to the unbalanced initial assignment, and the authors do not explainhow this was achieved.Subject selection biasOne of the primary concerns in attempting to replicate the treatment effect outside theoriginal study is the ability to select similar children for treatment.The children studied by Lovaas were not randomly sampled from the population of childrenwith autism.  Rather they were referred to the treatment program following clinical diagnosisin an affiliated, but independent institution.  Therefore, they may not represent a cross-section of children with autism.BC Office of Health Technology AssessmentAutism and Lovaas treatment22Of greater concern is whether Lovaas provided experimental treatment to ‘high functioning’children with autism, 47% of whom achieved normal functioning.  In other words, were theenrolled children representative of the population or were they selected consciously or semi-consciously by the referring centre because they were seen to have exceptional potential?Schopler et al 30 ist characteristics of the experimental treatment group that, in their words“skewed (it) toward relatively high-functioning autistic children”:i) Lovaas used a prorated mental age to “exclude many subjects with intellectualfunctioning higher than the profoundly retarded range” 21 (p4)that Lovaas suggestedwas the only group excluded.ii) Lovaas only included children between 40-46 months if they had echolalia, “asymptom widely recognized (also by Lovaas 1981) 46 as a characteristic of autisticchildren with a better prognosis”.  Lovaas notes, in response to Schopler,31 that 4children met these criteria for a better prognosis, but they were equally distributedamong the experimental and control groups.(iii) Lovaas selected children with an IQ of 63, “considerably higher than any random sampleof autistic children”.30  Schopler also claims that Lovaas used IQ measurements thatunder-estimate IQ.  Lovaas and Schopler disagree regarding normal epidemiological IQlevels, and whether the Lovaas experimental group were representative.Mesibov 39 raises similar concerns about subject selection.  He states:“Concerns about the representativeness and comparability of the sample group are raisedbecause (a) different cut-off ages were used for ecolalic and mute children, and the authorsdid not state how these ages were selected; (b) the control group had fewer higherfunctioning clients that one would expect in groups of this size: typically, 20% to 30% ofpeople with autism are higher functioning, irrespective of the services that they receive;and c) different testing protocols were used for clients in different groups.” 39 (p380)Boyd (1998) 32 provides the most cogent and unanswered criticism of selection bias found inLovaas 1987.  Boyd noted that the sex ratio in control group 1, the group for which there ismost outcome data (McEachin 1993) 22 does not reflect the sex ratio found in a standardautism population.  The sex ratio, accepted by both Boyd and Lovaas, is 3 or 4:1, boys to girls.Of the 19 children in each group, there are 3 girls (16%) in the experimental group and8 girls (42%) in control group 1.  The ratios are 5.3:1 for the experimental group, and1.4:1 for control group 1.  Boyd goes on to cite evidence, not refuted by Lovaas or hiscolleagues, that “a number of studies have shown autistic females to have lower IQ scoresthan males”.39 (p212)  One study, Volkmar et al (1993), cited by Boyd showed: “Significant sexdifferences were evident with IQ but not on measures of autistic symptoms, with girls havingsignificantly lower IQ scores.  The sex ratios for IQs below 35, 35-69, 70 or above were,respectively, 2.84:1, 3.41:1, 2.7:1.” 32 (p212)Boyd argues that, when the general population sex proportions are known, the controlgroup sample should be compared to that population proportion.  Using this assumptionand a simple chi-square test statistic, he demonstrates that control group 1 is statisticallysignificantly different from a population of children with autism.  Indeed, he argues(forcefully, in the opinion of the BCOHTA researchers) that the much higher percentage ofgirls in the control group could account for most of the difference in the experimental andcontrol groups.BC Office of Health Technology AssessmentAutism and Lovaas treatment23Boyd 32 and Smith & Lovaas 33 agree that the effect of sex on the outcome of active treatmentis unknown.  However, Smith and Lovaas provide no adequate explanation for how theydealt with the biased control group 1.  Furthermore, Smith and Lovaas argue incongruouslythat the differences between the experimental and control groups were not significantlydifferent, ignoring the need to establish the cell frequencies based on population sex ratios.Smith and Lovaas compound the inadequacy of their response by noting individual effects,not group effects.  They state: “the girls with autism who received intensive treatment in theLovaas (1987) study fared quite well.  Two of the three achieved ‘normal functioning’ at age7, and at age 12 (McEachin et al, 1993).” 33 (p343)  In other words, the fact that two girlsachieved normal functioning does not diminish the importance of recognizing that, onaverage, girls do not do as well as boys.In summary, the issue of whether the Lovaas (1987) experimental treatment sample wasrepresentative is unanswerable without additional research, using random selection ifpossible, or at minimum, random treatment allocation.  At this point it is only possible tosay that the best outcome children made very significant gains over the course of the study.3) The study should use standard intervention protocols that capture a wide rangeof skills and symptoms, under both laboratory and ‘real life’ situationsThis criterion is met by Lovaas et al and has not been at issue in the published literature, norfor the BCOHTA analysts.  The Lovaas method of intensive behavioural intervention relieson a published manual providing extensive details of treatment definitions and protocols.46Therapists providing Lovaas treatment can also receive extensive theoretical and practicaltraining.  In addition, the Lovaas method manages a wide range of intellectual and socialskills in various residential, community and home situations.  Indeed, the Lovaas method iscredited with helping to move therapy beyond the clinic and the laboratory into real lifesettings such as homes and the community.  It should be noted, however, that while criterion3 is clearly met, criterion 6 (details of the application of that method) is not.4) The study should make use of outside evaluators who are not invested in theoutcome of the researchThe extent to which outside evaluators were used by Lovaas at baseline has remainedcontroversial.  Gresham and McMillan note that, at intake, 5 children were not evaluatedby outside evaluators, but by the parents.36Smith and Lovaas claim outcome assessment at school entry “the evaluators were hired bya psychologist unaffiliated with the UCLA project and told that they would be working ona study of child development.  As an additional precaution against their discovering thepurpose of the study, the evaluators assessed equal numbers of children with no history ofclinically significant behavioral disturbances.” 35 (p202)Gresham and McMillan note that outcome in terms of school performance and behaviourwas reported by parents and teachers, not by outside evaluators.  Outside evaluation waslimited to IQ testing of the 9 children from the active treatment group who achieved thehighest level of functioning.36BC Office of Health Technology AssessmentAutism and Lovaas treatment245) The study should assure high compliance with the defined treatment protocol toensure that the intervention is actually and consistently implemented as designedTreatment integrity, also known as reliability of treatment implementation, refers to theextent to which a treatment was actually applied.  Gresham and McMillan provide the mostdetailed critique of this aspect of the Lovaas 1987 study.  They state that the “failure tomonitor the degree to which treatment procedures were implemented as planned acrosstherapists and subjects calls into question the internal validity of these findings”.19Gresham and MacMillan 19 consider the lack of detail in actual adherence to treatment protocolsas a significant problem in terms of ‘construct validity’, that is, the causal explanation linkingindependent treatment variables to dependent variables in terms of autism outcome.  Clearly-defined treatment activities are essential if treatment effect is to be distinguished fromextraneous variables influencing the outcome of children with autism.Gresham and MacMillan observe, “The (Lovaas) 40-hour-or-more treatment protocol wassupposedly responsible for dramatic behavior change, and the 10-hour-or less treatmentproduced no effects.  This suggests that the dramatic treatment effects for the experimentalgroup were due primarily to the amount or dose of treatment rather than to the discrete trialtraining procedures per se.  Without integrity data, we cannot know what aspects of the(Lovaas) treatment supposedly make it work”.19 (p196)BCOHTA researchers concur with Gresham and MacMillan.  In neither the Lovaas 1987study report, nor in the subsequent discussions do Lovaas et al provide details of whichelements and how much of the therapeutic intervention individual children received.  Alsoomitted are details of the type and amount of alternative (perhaps pharmaceutical) therapiesreceived by the experimental group.  For example, the study authors do not detail howmany children in control group 1 received 10 hours per week of treatment and how manychildren received less; nor do they detail what additional community treatment the childrenmay have received.Replication of the Lovaas (1987) treatment benefit requires replication of the comprehensivetreatment model including parents, therapists and pre-school institutions.  This could beextremely problematic, as Gresham and MacMillan observe.  “Absent the same level ofexpertise, training, and close supervision, one can only wonder whether favorable outcomes,let alone ‘normal functioning’ can be achieved at sites being required to provide the [Lovaas,UCLA treatment program].” 37 (p10)  Further research is clearly needed to refine the treatmentinstrument and determine a relationship between specific treatment activities, duration andchild benefit.Of equal concern in attempting to reconstruct the treatment effect is the timing of treatment.Lovaas explains that the experimental group, on average, began treatment approximately6 months earlier than the control group (34.6 months versus 40.9 months).  However, nodetails were provided regarding the time when individual children started treatment; howlong they had treatment; or when follow-up assessment occurred.  It is possible that a childbegan treatment at age 2, had treatment for two years, but was not examined in school until3 years later.BC Office of Health Technology AssessmentAutism and Lovaas treatment25In summary, despite the detailed material regarding the Lovaas method, the research reportsprovide virtually no details regarding the actual application of that material.  In fact, theabsence of description of the amount, quality, and consistency of the applied behaviouralanalysis techniques is one of the most striking aspects of Lovaas and McEachin’s work.6) The study should use longitudinal designs that evaluate treatment effects,both during the treatment itself, and at set points after the intervention hasbeen accomplishedThe Lovaas study meets this criterion in that it is a longitudinal design which evaluatestreatment effects.  At issue, however, is how the treatment effect was evaluated.Several authors strongly criticize the way that Lovaas evaluated treatment effect.  Schopleret al see an excessive use of classroom placement and IQ as outcome measures.  They state:“Classroom placement may have less to do with changes in the child than with the policiesof the school system toward special-needs children”.30  Lovaas et al respond that placementis determined independently in this geographical area.  If children had autism, then theschool district would have placed them in separate schools.Schopler et al also note that “Improvement on IQ measures may reflect improvement incompliance rather than in cognitive functioning.  Higher scores on follow up IQ tests willthen reflect improved test-taking skill rather than improved intellectual functioning.” 30 (p162)Mesibov similarly notes that “Many skills required for normal functioning have not beenmeasured by McEachin et al (1993).  They did not report on the students’ social interactions,friendships, conceptual abilities, and social communication skills, skills likely to differentiatechildren with autism from their peers without handicaps.  These are important aspects of theautism syndrome and deserve more scrutiny in further studies.” 39 (p380)Mundy similarly states: “In summary, the interpretation of the McEachin et al (1993) data setshould be done cautiously and be constrained by the limits of the outcome measures.  Theoutcome measures used may not have been of sufficient breadth or specificity to addressthe issue of whether the best-outcome group displayed remission of autistic symptoms ordisplayed symptoms typically presented by high functioning children with autism.” 40 (p383)Gresham and MacMillan 37 raise different concerns regarding the measurement instrumentsused in Lovaas 1987, particularly at baseline.  While it is outside the expertise of BCOHTAresearchers to interpret the validity of this critique or the responses by Lovaas et al, thesecomments are included because they raise important issues regarding the baselinecharacteristics of the study population.Gresham and MacMillan 19 (p190) provide a technical critique (similar to that of Schopler 30)of the use of a scaling procedure termed the ‘prorated mental age’ (PMA).  They challengeLovaas et al to explain several details of the procedure, so that its validity might be assessed.Gresham and MacMillan 19 note that the original trial report does not provide sufficient detailto determine individual children’s ages, mental ages and how these were combined with theIQ tests.BC Office of Health Technology AssessmentAutism and Lovaas treatment26Lovaas (1987) only reports averages for each of the groups.  Gresham and MacMillansummarize their concerns: ‘we simply do not know how to interpret the posttest IQ resultsgiven that posttest measures primarily were scaled as deviation IQs and pretest scores werebased on a PMA using a potpourri of development scales”.19 (p189)Several authors 30,31,37,40,44 discussed the relative merits of various tests of psycho-socialdevelopment.  Again, these issues are beyond the expertise of BCOHTA researchers,who confine comment to agreeing with the more generally valid methodologicalobservation made by Gresham and MacMillan that: “all pretest measures should haveused the same scale and that all posttest measures and follow-up measures should haveused the same scale”.36 (p221)4.3  Summary of critical appraisal of Lovaas et al (1987)The BCOHTA critical appraisal agrees with other commentators that the study by Lovaas(1987) 21 and McEachin (1993) 22 suffers from several major methodological limitations.These limitations include non-random assignment of children to treatment and control, anunrepresentative control group 1 in terms of sex ratio, and inadequate documentation oftreatment integrity.In addition, different assessment tools were used on the children at baseline and administeredunder non-standard conditions; and follow up measures limited to IQ tests may have missedresidual problems.  As a result, the BCOHTA reviewers conclude that it is very difficult torely on the effectiveness claims made by Lovaas et al.BCOHTA reviewers and other commentators also raise serious concerns regarding theexternal validity (reproducibility of treatment effect in contexts outside the original studysetting) of the Lovaas study.  The principle concern is the extent to which Lovaas therapyrequires restriction to a higher functioning sub-group of children with autism.  Lovaas admitsthat the poorest functioning 15% of children referred were excluded, while Schopler et alestimate that applying the Lovaas inclusion criteria in their context would eliminate 57%of referrals.30 (p163)Of equal concern in reproducing the Lovaas treatment effect is that, while the Lovaasintervention model offers extensive documentation, the Lovaas report provides almostno details of actual treatment activities, of staff training, or of quality control.BC Office of Health Technology AssessmentAutism and Lovaas treatment275.   CRITICAL APPRAISAL OF OTHER PRIMARY STUDIESThis section appraises three additional studies reporting primary data on early (pre-school)intensive therapy, with a control group.  The first two, Birnbauer and Leach (1993) 23 andSheinkopf et al (1998) 24 both evaluated the effectiveness of Lovaas therapy.  The third,Ozonoff et al (1998) 25 evaluated the effectiveness of a home-based therapy program.The following critical appraisal of these studies utilizes the same NIH conference criteriaapplied in the previous section (listed in section 2.2 above).  Comments from other publishedsystematic reviews with critical appraisals are added.5.1  Birnbauer and Leach (1993)DescriptionThis was a prospective study that treated 11 children, but only reports on 9 (5 boys), with amean age of enrollment of 39 months, all with DSM-III diagnosis of autism, or pervasivedevelopment disorder.  The non-treated control group consisted of 5 boys who were‘younger’, but ‘matched’ on all other variables.At intake, children were assessed in terms of IQ, adaptive behaviour, personality, andparental stress.  They were repeated every 12 months during therapy.  Final outcome datawere collected at 24 months.The authors did not conduct any group analysis.  They report that, after 24 months, 4 of the 9treated children had non-verbal IQs of 89 or higher; the language levels of the treated groupdoubled that of controls.  All children in both groups continued to have symptoms of autismat the end of 2 years.  For example, both groups continued to show marked levels ofstereotypic behaviour and poor toy-play.  Of the 5 controls, one made marked improvementand 4 made mild to moderate improvement.1) The study should compare various approaches to treatmentNo comparison of treatment options.2) The study should involve random assignment to different treatment conditionsNot randomized.  In fact, the sample size is so small and the method of determining a controlgroup so weak, this study more closely resembles an uncontrolled case series than acontrolled experiment.3) The study should use standard intervention protocols that capture a wide rangeof skills and symptoms, under both laboratory and ‘real life’ situationsThe intervention utilized the Lovaas treatment manual,46 and reports a mean of 29 hoursper week of therapy at home.  However, the therapy was carried out by a variety of trainedvolunteers.BC Office of Health Technology AssessmentAutism and Lovaas treatment284) The study should make use of outside evaluators who are not invested in theoutcome of the researchThe authors state that all the children had clinical diagnoses of autism made by an outsideagency.  The authors provide no details regarding diagnostic tests, criteria or reliability.The standardized tests and behavioural assessments during the study period were carriedout by examiners blinded to patient assignment.  Behavioural assessments were tested forinter-rater reliability.5) The study should assure high compliance with the defined treatment protocol toensure that the intervention is actually and consistently implemented as designedNo description of compliance; no testing of treatment application.6) The study should use longitudinal designs that evaluate treatment effects,both during the treatment itself, and at set points after the intervention hasbeen accomplishedThe study makes no effort to relate the amount or quality of therapy to clinical outcome.It is prospective, but limited to the time of active treatment.  The study does not report onindividual patient trends over the course of therapy.Other systematic reviews with critical appraisal:In her systematic review, Rogers provides this summary comment regarding Birnbauer andLeach:“This article represents an independent replication of Lovaas’ model, and reports positiveoutcomes associated with the treatment.  However, some difficulties limit its usefulness asa replication.  The children received less than half of the number of treatment hours thatLovaas reported.  There was no information regarding the fidelity of the treatment modeldelivered by the therapists.  No long-term follow up data were reported, and there were nostatistical analyses of group differences on pre- and post-treatment measures.  The high-improvement group was defined by their non-verbal IQ scores in the 85 or above range.However, non-verbal IQ is a problematic outcome measure.  Positive changes in non-verbal IQ do not necessarily reflect widespread gains in children with autism because themeasures tap an area of strength in autism - visual-perceptual reasoning - and do notmeasure areas of definitive functioning.  There was also a considerable amount of missingIQ data.  Two of nine treated children and three of five control children had no IQ data atall, and full IQ protocols were reported on only four of nine treated children and one offive controls.  Moreover, the mean developmental rate of the treated group in the area ofadaptive behavior was actually below that of the control group.” 26 (p173)In his systematic review, Smith notes:“Children … received less intensive therapy than children at UCLA (an average of 18-25 hr/week, instead of 40).  Also, service providers received less supervision fromsenior staff and, at one site (Birnbauer et al) appeared to have substantially lessexperience, typically leaving after 13 weeks of providing 2.5 hours of treatment perweek (a total of 32.5 hours) … (T)he investigators reported that their group of ninechildren made significant gains, but when the results are expressed as IQ, it appearsBC Office of Health Technology AssessmentAutism and Lovaas treatment29that the group showed little improvement  … (I)t is evident that, at least under somecircumstances, behavioral analytic early intervention programs may fail to yield clearlypositive results.” 48 (p38)BCOHTA commentThe BCOHTA researchers noted that this study report is remarkably similar to Lovaas 1987 21and McEachin 1993 22 in that, while both cite references to Lovaas treatment with itsextensively documented protocols and manuals, they provide remarkably few details onwhat therapy actually took place.  The lack of validation of treatment integrity underminesclaims of a causal link to overall outcome and limits the ability to replicate the study in anindependent context.Summary of Birnbauer and LeachThis study is considered too small, too short and too methodologically weak to providestrong evidence for or against the effectiveness of Lovaas therapy.  Particularly noteworthyis the fact that it does not validate the remarkable success rate reported in Lovaas 1987 21,which reported that half of subjects achieve ‘normal functioning’.  In fact, none of thechildren in the study reported by Birnbauer and Leach functioned in the normal range onany parameter, in some cases following 24 months of therapy.5.2  Sheinkopf and Siegel (1998)DescriptionThis is the second published study, along with Birnbauer and Leach, which used a controlledstudy methodology to assesses independently Lovaas treatment effectiveness.24  This is aretrospective study of 11 children, all diagnosed with autism or pervasive developmentaldisorder, with a mean age of 33 months and a mean IQ of 63.The authors did not use a prospective, experimental design.  In fact, the researchers had norole in group assignment at all.  Parents ‘independently’ chose to instigate Lovaas therapy.The researchers then ‘selected’ these children retrospectively for inclusion from a largerlongitudinal study.The study report provides no indication of the total number of children who wereindependently started on Lovaas therapy, to what degree, or how many of this larger setformed the subset for inclusion.  The active treatment children (receiving Lovaas treatment)were matched pair-wise with community controls.Study findingsThe treatment group had a mean IQ 25 points higher than the control group (mean of 90versus mean of 64; p =.01).  All 10 children in the treatment group with outcome dataimproved on IQ measures, usually by 20 or more points.  In the control group 6 childrenimproved by small amounts to moderate amounts, four had lower scores, and one stayedthe same.BC Office of Health Technology AssessmentAutism and Lovaas treatment30The authors found no group differences on number of symptoms; the treated group haddecreased symptom severity (p = .01).  Ten of 11 children in the treatment group and 11/11in the control group still had a diagnosis of autism at the end of therapy.  No information wasprovided regarding language development, adaptive behaviour, academic or socialfunctioning.1) The study should compare various approaches to treatmentNo comparison of intensive treatment alternatives.  The control group received ‘community’therapy, poorly defined.2) The study should involve random assignment to different treatment conditionsNo random assignment.  This was not an experimental design, but a retrospectiveobservational study.  Moreover, as set out, the method of control group construction isincomprehensible: “Matching was accomplished by reverse serial selections of any casematching an index case, with controls drawn from a database of approximately 1,000children surveyed from most recent cases, backward.”3) The study should use standard intervention protocols that capture a wide rangeof skills and symptoms, under both laboratory and ‘real life’ situationsThe intervention is described as consisting of 1:1 instruction based on the Lovaas manual 46and the supervision of one of three community therapists and parents.4) The study should make use of outside evaluators who are not invested in theoutcome of the researchThe initial diagnoses and assessments were made by the primary researchers for all children.The authors note that the initial data on the children was collected prior to the researchers’awareness that the parents in the treatment group had, presumably independently, decided tobegin Lovaas therapy.  Post-treatment assessments were also made by the researchers.  Nostatement is made on the issue of blinding final outcome assessments.5) The study should assure high compliance with the defined treatment protocol toensure that the intervention as designed is actually and consistently implementedThe report describes use of Lovaas behavioural treatment in home and regular communitytreatment; mean 27 hours per week for a mean of 21 months.  However, there is nodocumentation of treatment fidelity.  Data regarding treatment was gathered retrospectivelythrough “phone interviews with one parent for each child.” 24 (p18)The issue of treatment fidelity is made even more problematic in terms of training andquality control, as “(t)he authors of (the) paper were not involved in treatmentimplementation.”All children also received a wide range of community and professional services, “in an equalamount.”  The presumed difference leading to the subject benefit was that the treatmentgroup received a mean of 20 hours per week of one-to-one treatment.  The control groupreceived a mean of 11 hours of treatment (not necessarily one-to-one).BC Office of Health Technology AssessmentAutism and Lovaas treatment316) The study should uses longitudinal designs that evaluate treatment effects,both during the treatment itself, and at set points after the intervention hasbeen accomplishedThe study used only IQ measures (mainly non-verbal), number of symptoms, and severity ofsymptoms.  Rogers’ remarks 26 on the difficulties posed by non-verbal IQ measures are notedin the commentary on Birnbauer and Leach (p29 above).Comments of other systematic reviews with critical appraisal:In her systematic review Rogers states: “As with the Birnbauer and Leach (1993) study,there was some evidence of reduced severity of symptoms of autism in the experimentalgroup after two years of treatment, but virtually all children continued to meet criteria forautism.” 26 (p174)By contrast, the systematic review by Smith 48 provides no critique of the study methodologyused by Sheinkopf and Siegel.  Instead he uncritically cites the findings by Sheinkopf andSiegel as showing IQ benefits for the treatment group using Lovaas therapy.The systematic review by Green, similar to Smith’s, accepts the findings of Sheinkopfand Siegel as providing “evidence that intensive behavioral intervention increases theintellectual functioning (as measured by standardized, objective tests) of many youngautistic children”.28 (p36)  She notes, however, that the study provides no details regardingthe treatment applied, and that there is no evidence that any children achieved normal ornear normal functioning.Additional BCOHTA commentThe study uses a very weak, partially retrospective, observational study design.  Althoughproviding some support of benefit from the Lovaas method, Sheinkopf and Siegel 24 didnot report any children as ‘normal’ or approaching normal functioning.  Indeed, owing tothe virtual absence of description of treatment effect or of central co-ordination of theintervention, it is impossible to determine any causal inference relating to the limited gainsreported for these children.5.3  Ozonoff and Cathcart (1998)This is a study of 11 children treated with a 10-week, home program service based on aform of therapy termed “Treatment and Education of Autistic and related Communication-handicapped Children” (TEACCH).25  TEACCH is a long-established special educationagency formed in the 1960s and located in North Carolina, serving children with autism.Most children receiving the TEACCH program are involved in classroom settings.49 (p46)TEACCH, in contrast to Lovaas therapy, considers its method involves “structured teaching”as opposed to behavioural therapy.  As Smith explains: “In contrast to behavioral treatment,Project TEACCH is aimed primarily at designing sheltered settings that help children makeuse of the skills they already possess, rather than at helping children to enter more ‘normal’or ‘typical’ settings.” 49 (p46)BC Office of Health Technology AssessmentAutism and Lovaas treatment32In the Ozonoff study, the home program service was added to the regular day-treatmentprograms attended by all the children.25  Th  regular day-treatment programs in this part ofthe USA are described by the authors as consisting of the ‘discrete trial method’ of appliedbehaviour analysis used in Lovaas therapy.  Children in this study thus were receivingintensive therapy following two different treatment orientations.The home program involved 10 treatment sessions, in the clinic and home, designed to assistparents in becoming co-therapists.  All families received treatment during the same 4-monthperiod.  Children were age 2 - 6 (2 girls and 9 boys).FindingsChildren in the treatment group gained on 4 of 7, and on total score in the Psycho-educational Profile - Revised developmental test.  The treatment group gained 9 months onthe developmental scale during 4 months of therapy.The study seems to have been published too recently for consideration in other systematicreviews of controlled trials of early intensive treatment.  Therefore the critical appraisalcomments are almost exclusively from BCOHTA reviewers.1) The study should compare various approaches to treatmentNo comparison of home-based approaches.  In this instance, the researchers compared theaddition of home-based treatment to regular school-based treatment.2) The study should involve random assignment to different treatment conditionsNot randomized.  The study enrolled the first 11 families to respond to an advertisement tothe treatment group, and the next 11 to the control group.3) The study should use standard intervention protocols that capture a wide rangeof skills and symptoms, under both laboratory and ‘real life’ situationsThe home-based program, although based on TEACCH philosophy, “did not follow aspecific time-line, packaged protocol or manual, but emphasised individualized, structuredteaching.” 25 (p28)4) The study should make use of outside evaluators who are not invested in theoutcome of the researchThe study relied exclusively on internal evaluators, aware of treatment allocation, todetermine intervention effect.  The study does not report the method of clinical diagnosis.5) The study should assure high compliance with the defined treatment protocol toensure that the intervention is actually and consistently implemented as designedParents were not taught a pre-defined set of skills.BC Office of Health Technology AssessmentAutism and Lovaas treatment336) The study should use longitudinal designs that evaluate treatment effects,both during the treatment itself, and at set points after the intervention hasbeen accomplished.The study appropriately used the same test for pre- and post-intervention assessments.(Psycho-educational Profile - Revised developmental test).  The study was limited toassessment of cognitive and developmental parameters.  It did not examine adaptivebehaviour, behaviour problems, or social functioning.Additional BCOHTA commentThe BCOHTA authors agreed that this study is worth considering because, although theintervention period is only 10 - 12 weeks and it suffers from many methodological flaws,it is a prospective controlled trial using overall outcomes that studied an intensive, home-based treatment alternative to Lovaas therapy.The authors’ conclusion seems appropriately conservative given the methodologicalweakness found: “The results of this study suggest that auxiliary home interventions increasedevelopmental functioning in young autistic children, above and beyond gains due to school-based services.” 25 (p31)5.4  Summary of critical appraisal of corroborative researchThe effectiveness claim found in Lovaas et al (1987),21 that half the children achievednormal or near-normal development and placement in schools, remains uncorroborated byindependent research.In fact, while improvement in IQ and autism symptoms were described for most children,none of the children in two other controlled trials (Birnbauer and Leach;23 Sheinkopf andSiegel 24 ) achieved a normal development stage by the end of therapy.  It should be noted,however, that in both these instances the children received about half (20 hours per week)the intervention intensity described in the original Lovaas study (1987).21There are no controlled trials comparing the Lovaas method with alternate intensive, one-on-one therapies.  The controlled study by Ozonoff, 25 alth ugh short and small-scale, providesat least some evidence that alternate, home-based programs, in this instance emphasizingparent teaching as opposed to applied behavioural analysis, need further evaluation bothindependently and in comparison with Lovaas treatment.The published effectiveness evidence does not include any studies which examined overalloutcome following early diagnosis using a screening manoeuvre.  That is, studies to datehave not followed a population of children diagnosed ‘early’ in the natural history of thecondition to determine both the benefits from early treatment and the costs associated withfalse-positive and false-negative diagnostic labelling.The need for adequate outcome trials will increase as screening tools are proposed.  To date,the cost and benefit of screening tools remain completely unknown, although the literaturedoes show preliminary work toward establishing the cost and benefit of early (age 2) versuslater (age 3) diagnosis.BC Office of Health Technology AssessmentAutism and Lovaas treatment34For example, Lord 50 followed for one year a cohort of 30, age-2 children referred forpossible autism to determine how many received false-positive and false-negative clinicaldiagnoses.  Other researchers have conducted similar analyses of the validity of earlydiagnostic programs.51,52  None of these studies, however, is linked to treatment or tooverall outcome.Support for ‘early’ therapy should therefore be directed toward early treatment of childrenknown to have autism, and not toward therapy of children receiving an ‘early’ diagnosis.Rogers similarly concludes: “The hypothesis that age at start of treatment is an importantvariable in determining outcome has tremendous implications for the field and needs to betested with methodologically-rigorous designs.” 26 (p176)BC Office of Health Technology AssessmentAutism and Lovaas treatment356.   DISCUSSIONThe published effectiveness evidence on overall outcome from intensive behavioural therapyfor pre-school aged children with autism gives rise to two problematic issues.  The first is theinconsistency between the Lovaas effectiveness claims and subsequent accounts of similartreatment success.  The second is the degree to which the effectiveness claims have beenadvanced despite these inconsistencies.6.1  Lovaas effectiveness claimLovaas 1987 21 (with McEachin et al 1993 22 ) in essence claim that by applying their methodfor 40 or more hours per week to higher-functioning children with autism (that is, omittingthe most retarded 15%), approximately half of the children will achieve IQ measures in thenormal range and placement in regular schools, and will also be described by their teachersand peers as indistinguishable from other children.Although providing insightful self-criticism of his study methodology and the need foradditional research, Lovaas nevertheless uncritically promotes the assertion that his methodvirtually results in normal functioning.  The following exemplifies this stance:“Like any other study, ours needs to be replicated by independent investigators,and it could have been improved in several ways.  For example, we assignedsubjects to groups on the basis of therapist availability rather than on the basis ofa more arbitrary procedure (such as tossing a coin) because parent advocacy groupsprotested arbitrary assignment.  However, arbitrary assignment might haveprecluded the suspicion that we somehow biased our groups.  Also, further studiesmay show how to simplify or shorten the intervention and how to increase theproportion of autistic children who attain normal functioning.” 31 (p167)Mesibov raises the principal concern with this approach: “… readers might jump to theconclusion that the children have been cured.  This has been an unfortunate consequenceof other presentations and studies published by these authors.  Although their results areimpressive, they fall far short of demonstrating normal functioning.” 39 (p380)In addition, Rutter, an influential researcher in the autism field, concurs that the effectivenessclaim is overstated:“The [Lovaas (UCLA) treatment program] claims have given rise to controversy,sometimes heated, but existing empirical evidence does not provide a resolution.There are reasons for being cautious about the acceptance of these strong claims(if only because of the uncertainties of diagnosis in very young children and becausethe claims of ‘cure’ run counter to both clinical experience and what might be expectedfrom prevailing theories) … doubts about the claims should lead to further studies totest them rather than a dismissal on the basis of existing evidence.” 53BC Office of Health Technology AssessmentAutism and Lovaas treatment36The BCOHTA researchers accept the later, more moderate statement by the authors.  In thispassage, Smith, McEachin and Lovaas summarize points of agreement with critics of theLovaas 1987 study:“First, our best-outcome subjects appear to have made significant gains.  Second,we have made a plausible case for attributing these gains to our treatment insteadof extraneous factors such as spontaneous recovery.  Third, this finding needs tobe replicated because no single study by itself can give conclusive evidence on theefficacy of a treatment.  Finally, replications should not simply rehash what hasalready been done.” 44 (p385)6.2  Failure of corroborative effortsPublished controlled studies, as documented in section 5 above, do not report childrenachieving normal functioning as defined by Lovaas 1987 21 (McEachin 1993).22Furthermore, uncontrolled studies, although a weaker form of evidence than controlledstudies because they do not account for the development process outside therapy, similarlydo not support conclusions of ‘normalization’ through Lovaas therapy.For example, Anderson et al,54 who applied the Lovaas treatment method more rigorouslythan any but the original trial itself, concluded that while 6 of 14 children studied showedIQ gains of 20-22 points over 2 years, all children continued to need special educationservices and none were mainstream in a regular kindergarten or school.Two other uncontrolled studies, Harris et al (1991) 55 and Fenske et al (1985),56 reachedsimilar conclusions.  Harris et al report on a centre-based intensive behavioural therapyprogram for nine children after one year, and conclude that “all treated children continuedto demonstrate impairments after treatment”.26 (p175)  Fenske et al reported on nine pre-schoolchildren after 2 years of intensive treatment at the Princeton Child Development Centre.Rogers notes that, “no information was given about the children’s level of functioning inpublic school classes or any other outcome variables.” 26 (p175)Systematic reviews with critical appraisal similarly fail to find support for Lovaas’effectiveness claims:a) Rogers: “[Studies] did not demonstrate the level of improvement in multiple areas offunctioning or the sustained long-term effects of the treatment that Lovaas reported.The field awaits a full, independent replication of the Lovaas study.” 26 (p176)b) Tregear et al (ECRI * ): “Although it appears possible to improve some aspects offunction in autistic children, it is not clear that any one program is more effective thananother.” 27c) Howlin: “For the present, Lovaas programme clearly confirms the power of behavioralinterventions.  The true extent of the benefits, however, still requires greater explorationand longer term evaluations, covering many other aspects of functioning are needed ifthe true cost-effectiveness of the time, effort and energy expended by families is to beadequately assessed.” 7 (p61)                                                          * ECRI is a non-profit health-services research agency whose reports are not in the public domain but areavailable for purchase.  <http://www.healthcare.ecri.org>BC Office of Health Technology AssessmentAutism and Lovaas treatment37d) Green, in the systematic review that conducted the least critical appraisal, draws thestrongest conclusions in favour of Lovaas treatment.  She states that “applied behavioralanalysis is the treatment of choice, it should have at least 30 hours of 1:1 therapy perweek by individuals with extensive training, and it will result in very significant costsavings over the lifetime of individuals with autism.” 28 (p42)e) Smith, a co-author of the McEachin study along with Lovaas, provides a somewhatcautious interpretation of the research evidence in terms of achieving normal functioning:“It is encouraging that debates over how much and what kind of early interventionchildren with autism should receive have largely replaced debates over whether suchintervention merits particular attention at all.” 48 (p45)However, he also states in relation to his study (McEachin et al 1993) 22 “… someoutcome studies indicate that a major breakthrough may have occurred.  Nevertheless,these studies require replications with improved research methodologies and, even ifinterpreted in the most favorable possible light, reveal substantial shortcomings in theinterventions under investigation.  Therefore, much research remains to be done.” 48 (p42)The absence of corroborative evidence of ‘recovery from autism’ does not devalue theeffectiveness of early, intensive and comprehensive treatment programs in achievingsignificant developmental gains.  However, Lovaas and his research colleagues have notlimited their effectiveness claims to achieving developmental gains.  Instead, they havepermitted and even fostered the premise that appears throughout the published literature,associating their therapy with a notion of achieving ‘normal functioning’ for as many ashalf a given population of children with autism.Despite the lengthy existence of the treatment report (available for almost thirty years) andthe dramatic effectiveness claim based on it (first aired thirteen years ago), while thepublished literature does offer some support for the Lovaas effectiveness claims throughcase reports of individual success (some well documented 28 (p36)), the literature has virtuallyno accounts of groups of children entering programs and ‘recovering’ from autism.6.3  Does the evidence point a way forward?As asserted at the outset, the aim of this review has been to evaluate the available evidenceregarding the effect of overall treatment programs on the long-term outcome of childrenwith autism.  The BCOHTA authors do not promote or discredit any of these complexprograms, developed in particular contexts for a wide range of social, professional andeconomic reasons.  The simple point to make here is that the outcome evidence does notsupport adopting a single entire program, whether the Lovaas or alternative models, basedon the likelihood of achieving normal children.It should be mentioned at this point that the source of current difficulties does not lieprincipally with the Lovaas and McEachin study reports, nor should the shortcomingsidentified here and elsewhere be taken as a dismissal of the primary research.On the contrary, this work deserves significant commendation.  The researchers havetaken steps, unprecedented at the time of the research, to ensure objective and independentBC Office of Health Technology AssessmentAutism and Lovaas treatment38assessments of their subjects, examined long-term outcome, and established control groupsunder what seem to be extremely difficult study circumstances.  It is also to be noted thatthe research and the treatment program described were ground-breaking in their orientationtoward early, intensive, home (non-institutional) treatment of children with autism.Because of the major methodological flaws, however, the findings of these researcherscannot be regarded as conclusive.  Rather, their contribution might have been expectedto fuel further hypothesis generation.  While this has occurred to some degree, the workhas to date been insufficient to bring the science to what might be described as anauthoritative level.Nevertheless, the notion that Lovaas therapy, if properly applied, can result in normalizationof children with autism has gone so far as to become the rallying point for planning autismservices (in BC and elsewhere), rather than, as might be expected, the central focus of furtherresearch aimed at examining an uncorroborated effectiveness claim.Yet it is the very absence of subsequent research, and, in that absence, extrapolation fromthis single provocative study to reach generalized conclusions regarding treatment effect,(at the extreme, the possibility of cure for half of children with autism) that has given riseto the present program-planning dilemma.  A scientific claim of effectiveness is being usedto force action, but the same scientific evidence is inadequate to provide any guidance onhow to proceed.Given the gulf that seems to separate the sides in this debate, how are the courts, policymakers, and most importantly the families whose daily lives are doubtless filled withunanswered questions, to make practical use of the available evidence?To help overcome the uncertainties, payers and program-planners may benefit fromfollowing two steps.  First, it will be important to identify an authoritative, objective andreliable means of monitoring the state of effectiveness evidence.  Second, in the absenceof scientifically-valid studies of overall autism programs, it seems prudent to revise existingautism services using incremental, not programmatic steps.6.4  Monitoring the state of effectiveness evidenceTo monitor the state of effectiveness evidence, it is suggested that program-planners andpublic-payers identify:i. a group that uses explicit methodology to appraise the validity of individual clinical trialsand to situate any valid effectiveness evidence in a rigorous evaluative framework; butii. a group that is not asked to create clinical practice guidelines.The American Psychological Association, Task Force on Psychological InterventionGuidelines 18 is one such group.  The Association has provided a clear methodology todetermine effectiveness evidence and to draw conclusions regarding the quality of thatevidence, without seeking to develop guidelines based on clinical opinion.With regard to Lovaas therapy, the Task Force group state that this program of therapy hasnot been established as efficacious according to their criteria: “To date, no comprehensiveintegrated intervention for autism has sufficient support to achieve either well-established orBC Office of Health Technology AssessmentAutism and Lovaas treatment39probably efficacious status partially because of concerns and problems with conductingrandomized assignment group designs and a lack of strong replication studies.” 18 (p142)Although not supporting a Lovaas program, the Task Force supports specific, non-comprehensive behavioural interventions (not dealing with overall outcome) as havingsufficient empirical support.  In other words, they support the position that applied behaviouranalysis has been shown to alleviate the symptoms of autism, but not the condition itself.The problems associated with developing guidelines, as opposed to clarifying the state ofscientific evidence, are exemplified by The New York State Department of Health, ClinicalPractice Guidelines.57The New York Guidelines give their strongest endorsement (Level A)* that “principles ofapplied behavioral analysis and behavioral intervention strategies be included as animportant element of any intervention program for young children with autism.” 57 (Ch4, p3)Although no actual evidence is specified, it seems reasonable to accept this recommendationis based on the strong evidence of symptomatic improvement, but not cure rates.However, the Guidelines go on to make several more Level A recommendations regardingfrequency, intensity and duration of  intervention that seem essentially to accept uncriticallyelements of the Lovaas treatment program.  They recommend that intensive behaviouralprograms include as a minimum approximately “20 hours per week of individualizedbehavioral intervention using applied behavioral analysis techniques (not including timespent by parents).” 57 (Appendix C, p1)On the other hand, they recommend excluding certain aspects of the initial Lovaas treatmentprogram that have become socially unacceptable.  For example, the authors recommendagainst use of physical aversives as part of the applied behavioural analysis program, butwithout noting that this may conceivably have been an essential component of the initialLovaas (1987) 21 treatment success.Nor do the Guidelines provide effectiveness evidence to support their recommendation foreither 20 hours per week of intensive therapy, or the requirement of a 1:1 therapist-childratio.  The guideline authors state that the precise number of hours should be adjusted forindividual children, but provide no guidance on what outcome measures should be usedto increase, decrease, or discontinue behavioural therapy altogether.57 (Appendix C, p1)This is not to direct particular criticism at the NY Clinical Practice Guidelines.  In fact, theyare similar to most guidelines developed by clinical experts, in that while acknowledgingthe limitations of the clinical effectiveness evidence, they tend to make recommendationsthat extend far beyond what the evidence itself will support.  As noted by nationalorganizations responsible for clinical practice development, unsubstantiated extrapolationfrom limited clinical effectiveness evidence is the major drawback of the clinical practiceguideline movement.58                                                          * Level A = Strong evidence is defined as evidence from two or more studies that met criteria for adequateevidence about efficacy and had at least moderate applicability to the topic, and where the evidenceconsistently and strongly supports the recommendation.57 (Ch1, p4)BC Office of Health Technology AssessmentAutism and Lovaas treatment406.5  Taking incremental, not programmatic stepsPublic payers faced with an urgent need, therapeutically, politically, and legally, to investin comprehensive, pre-school services for autism have no way of knowing which elementsof which form of early, intensive treatment are necessary and sufficient to best benefitchildren and their families.  Therefore, the only evidence-based direction forward requiresincremental, not programmatic steps.  Ultimately, specific behavioural therapies, showneffective, can be matched to local needs and the abilities of local therapists.Taking incremental steps based on available evidence may in fact already be the primaryforce shaping autism services.  Dawson and Osterling,59 for example, describe a convergenceof approach found in US programs.  They note that regardless of divergent philosophies oftherapy, eight independent programs have developed around a common model of early,intensive therapy for children with autism, which include parental involvement.The “elements of effective programs” Dawson and Osterling describe have been furtherdefined in a prospective study, funded by the Office of Special Education, US Departmentof Education.60  Based on expert opinion and survey results, they identify several keyelements shared by all “nationally known models or program” as the “earliest possible startto intervention; individualization of services for children and families; systematic playfulteaching, specialized curriculum; intensity of engagement; and family involvement.” 60 (p21)BC Office of Health Technology AssessmentAutism and Lovaas treatment417.   CONCLUSIONSThe authors of this systematic review concur with the observations of the AmericanPsychological Association Task force (6.4 above), and in addition draw the followingconclusions:1. The Lovaas et al (1987) 21 and McEachin (1993) 22 study, while methodologicallystronger than published reports of alternate comprehensive therapies, is inadequateto establish the degree to which this form of therapy results in children achieving‘normal’ functioning, however defined.2. There is insufficient effectiveness evidence to establish a relationship betweenthe amount (per day and total duration) of any form of early comprehensivetreatment program and overall outcome.3. Randomized trials of alternative early intensive treatment programs are ethicaland feasible to advance research knowledge.4. There is insufficient effectiveness evidence to conduct a cost-benefit analysisof early, intensive treatment programs in terms of ‘normalization’ of childrendiagnosed with autism.  It remains the case that without a soundly-baseddetermination of the extent to which the intervention may result in benefit,and the degree of any such benefit, cost-benefit analyses have no basis onwhich to proceed.BC Office of Health Technology AssessmentAutism and Lovaas treatment42BIBLIOGRAPHYAmerican Academy of Pediatrics, Committee on Children with Disabilities. Auditoryintegration training and facilitated communication for Autism. Pediatrics 1998;102(2 Pt 1):431-3.Anderson SR, Avery DL, Di Pietro EK, Edwards GL, Christian WP. Intensive home-basedearly intervention with autistic children. Educ Treat Children 1987;10(4):352-66.Atferdsanalytisk orientert habiltering av barn med autisme [Behavior analytic habilitation ofchildren with autism. Empirical basis and common misconceptions]. Tidsskrift for NorskPsychologroening 1998; 35(6): 537-44.Auditory integration training for behavioral, emotional, and hearing disorders. PlymouthMeeting (PA): ECRI; 1994.Autism: Part II. Harv Ment Health Letter 1997;13(10):1-4.Autism Services Interdepartmental Working Group. Implementation plan. Wellington, NewZealand: Ministry of Health; 1998.Autism Services Project Team. Autism services in New Zealand. Wellington, New Zealand:Ministry of Health; 1998.Autism spectrum disorders at 20 and 42 months of age: stability of clinical and ADI-Rdiagnosis. J Child Psychol Psychiatry 1999;40(5):719-32.Baer DM. Commentaries on McEachin, Smith, and Lovaas: quasi-random assignment can beas convincing as random assignment. Am J Ment Retard 1993 Jan;97(4):373-5.Baer DM, Wolf MM, Risley TR.  Some current dimensions of applied behavioral analysis.J Appl Behav Anal 1968;1:91-7.Bailey A, Phillips W, Rutter M.  Autism: towards an integration of clinical, genetic,neuropsychological, and neurobiological perspectives. J Child Psychol Psychiatry1996;37(1):89-126.Bailey DB Jr, Aytch LS, Odom SL, Symons F, Wolery M. Early intervention as we know it.Ment Retard Dev Disord Res Rev 1999;5:11-20.Beitchman JH, Inglis A, Schachter D. Child psychiatry and early intervention: III: thedevelopmental disorders. Can J Psychiatry 1992 May;37(4):240-4.Best L, Milne R, Wessex Institute for Health Research and Development. Auditoryintegration training in autism. Development and evaluation committee report; No. 66.Southampton, England: Wessex Institute for Health Research and Development; 1997.Best L, Milne R. Auditory integration training in autism. Bristol (UK): South and WestRegional Health Authority; 1997.BC Office of Health Technology AssessmentAutism and Lovaas treatment43Bettison S. The long-term effects of auditory training on children with autism. J Autism DevDis 1996;26(3):361-74.Birnbrauer JS. Empirically validated learning environments. Behavior Change1997;14(1):21-4.Birnbrauer JS, Leach DJ. The Murdoch early intervention programme after 2 years. BehavChange 1993;10(2):63-74.Bondy AS, Frost LA. The picture exchange communication system. Semin Speech Lang1998;19(4):373-88; quiz 389; 424.Boyd RD. Sex and bias: reply to Smith and Lovaas. J Autism Dev Dis 1998;28(4):344.Boyd RD. Sex as a possible source of group inequivalence in Lovaas (1987). J Autism DevDis 1998 Jun;28(3):211-5.Brasic JR, Barnett JY, Sheitman BB, Tsaltas MO. Adverse effects of clomipramine. J AmAcad Child Adolesc Psychiatry 1997 Sep;36(9):1165-6.Bristol MM, Cohen DJ, Costello EJ, Denckla M, Eckberg TJ, Kallen R, Kraemer HC, LordC, Maurer R, McIlvane WJ, Minshew N, Sigman M, Spence MA. State of the science inautism: report to the National Institutes Health. J Autism Dev Disord 1996 Apr;26(2):121-54.British Columbia. Ministry for Children and Families, British Columbia.  Ministry ofEducation. Autism action plan: implementation plan. Victoria (BC): Ministry for Childrenand Families; 1999.Bryson SE. Brief report: epidemiology of autism. J Autism Dev Disord 1996;26(2):165-7.Bryson SE, Smith IM. Epidemiolog of autism: prevalence, associated characteristics, andimplications for research and service delivery. Ment Retard Dev Disord Res Rev1998;4(2):97-103.Bryson SE, Clark B, Smith IM. First report of a Canadian epidemiological study of autisticsyndromes. J Child Psychol Psychiatry 1988;29(4):433-45.Buckley PF, Schulz SC. Clozapine and risperidone: refining and extending their use. HarvRev Psychiatry 1996 Nov-Dec;4(4):184-99.California.  Department of Developmental Services. Changes in the population of personswith autism and pervasive developmental disorders in California's Developmental ServicesSystem: 1987 through 1998. Sacramento (CA): California Health and Human ServicesAgency; 1999.Campbell M, Cueva JE, Hallin A. Autism and pervasive developmental disorders. In: WienerJM, editor. Diagnosis and psychopharmacology of childhood and adolescent disorders. 2nded. New York: John Wiley & Sons; 1996.  p151-92.BC Office of Health Technology AssessmentAutism and Lovaas treatment44Campbell M, Cueva JE. Psychopharmacology in child and adolescent psychiatry: a review ofthe past seven years: Part I. J Am Acad Child Adolesc Psychiatry 1995 Sep;34(9):1124-32.Campbell M, Schopler E, Cueva JE, Hallin A. Treatment of autistic disorder. J Am AcadChild Adolesc Psychiatry 1996 Feb;35(2):134-43.CAN Consensus Group. Autism screening and diagnostic evaluation: CAN consensusstatement. CNS Spectr 1998;3(3):40-9.Carter AS, Gillham JE, Sparrow SS, Volkmar FR. Adaptive behavior in autism. ChildAdolesc Psychiatr Clin North Am 1996;5(4):945-61.Cattell-Gordon D. Seminar on applied behavior analysis treatment for autism: Thedevelopment of an effective applied behavioral analysis program for a young child withautism: a parent's perspective. Infants Young Children 1998;10(3):79-85.Chabrol H, Bonnet D, Roge B. Psychopharmacology in autism. Encephale 1996;22(3):197-203.Connor M. A review of behavioural early intervention programmes for children with autism.Educ Psychol Pract 1998;14(2):109-17.Cook Jr EH, Leventhal BL. Autistic disorder and other pervasive developmental disorders.Child Adolesc Psychiatr Clin North Am 1995;4(2):381-99.Coucouvanis J. Behavioral intervention for children with autism. J Child Adolesc PsychiatrNursing 1997;10(1):37-46.Cox A, Charman T, Baron-Cohen S, Drew A, Klein K, Baird G, Swettenham J, WheelwrightS. Autism spectrum disorders at 20 and 42 months of age: stability of clinical and ADI-Rdiagnosis. J Child Psychol Psychiatry 1999;40(5):719-32.Dawson G, Osterling J. Early intervention in autism. In: Guralnick MJ, editor.  Theeffectiveness of early intervention.  Baltimore (MD): P.H. Brookes; 1997.  p307-26.Drummond MF. Methods for the economic evaluation of health care programmes. New York(NY): Oxford University Press; 1997.Eikeseth S, Lovaas OI. The autistic label and its potentially detrimental effect on the child'streatment. J Behav Ther Exp Psychiatry 1992;23(3):151-7.Elbing U, Rohmann UH. Evaluation of an intensive therapy program for treatment of severebehavioral disorders in mentally handicapped patients with autistic or psychotic behavior.Prax Kinderpsychol Kinderpsychiatr 1993 Sep;42(7):248-60.Elder JH. Behavioral treatment of children with autism, mental retardation, and relateddisabilities: ethics and efficacy. J Child Adolesc Psychiatr Nurs 1996 Jul-Sep;9(3):28-36.BC Office of Health Technology AssessmentAutism and Lovaas treatment45Fenske EC, Zalenski S, Krantz PJ, McClannahan LE. Age at intervention and treatmentoutcome for autistic children in a comprehensive intervention program. Analysis InterventionDev Disord 1985;5:49-58.Field T, Lasko D, Mundy P, Henteleff T, Kabat S, Talpins S, Dowling M. Brief report:autistic children's attentiveness and responsivity improve after touch therapy. J Autism DevDisord 1997 Jun;27(3):333-8.Findling RL, Grcevich SJ, Lopez I, Schulz SC. Antipsychotic medications in children andadolescents. J Clin Psychiatry 1996;57 Suppl 9:19-23.Findling RL, Schulz SC, Reed MD, Blumer JL. The antipsychotics: a pediatric perspective.Pediatr Clin North Am 1998;45(5):1205-32.Foxx RM. Sapid effects awaiting independent replication. Am J Ment Retard1993;97(4):375-6.Freeman BJ. Guidelines for evaluating intervention programs for children with autism.J Autism Dev Disord 1997 Dec;27(6):641-51.Friman PC, Lucas CP, Lubetsky MJ, Holttum JR. Behavioral treatment for autism.  J AmAcad Child Adolesc Psychiatry 1994;33( 9):1349-51.Gadow KD. An overview of three decades of research in pediatricpsychopharmacoepidemiology. J Child Adolesc Psychopharmacol 1997;7(4):219-36.Gillberg C. The long-term outcome of childhood empathy disorders. Eur Child AdolescPsychiatry 1996;5:52-6.Goodman JF. Combining traditional models of education with normal and disabledpreschoolers. Educ Psychol 1994;29(3):113-20.Grandin T. Brief report: response to National Institutes of Health report. J Autism DevDisord 1996 Apr;26(2):185-7.Green G. On valid inferences: comments on Weiss. Behav Interventions 1999;14:23-7.Greenhill LL. The use of psychotropic medication in preschoolers: indications, safety, andefficacy. Can J Psychiatry 1998;43( 6):576-81.Greenspan S, Love PF. Social intelligence and developmental disorder: mental retardation,learning disabilities, and autism. In: MacLean WE Jr., editor. Ellis' handbook of mentaldeficiency, psychological theory and research. 3rd ed. Mahwah (HM): Lawrence ErlbaumAssociates, Inc; 1997. p311-42.Gresham FM, MacMillan DL. Autistic recovery?: an analysis and critique of the empiricalevidence on the early intervention project. Behav Disord 1997 Aug;22(4):185-201.Gresham FM, MacMillan DL. Denial and defensiveness in the place of fact and reason:rejoinder to Smith and Lovaas. Behav Disord 1997;22(4):219-30.BC Office of Health Technology AssessmentAutism and Lovaas treatment46Gresham FM, MacMillan DL. Early Intervention Project: can its claims be substantiated andits effects replicated?. J Autism Dev Disord 1998 Feb;28(1):5-13.Gupta S, Aggarwal S, Heads C.  Dysregulated immune system in children with autism:beneficial effects of intravenous immune globulin on autistic characteristics. J Autism  DevDisord 1996 Aug;26(4):439-52.Guralnick MJ. The next decade of research on the effectiveness of early intervention. ExceptChild 1991;58(2):174-83.Harris SL. Behavioral and educational approaches to the pervasive developmental disorders.In: Volkmar FR, editor. Autism and pervasive developmental disorders. Cambridge:Cambridge University Press; 1998. p.195-208.Harris SL, Handleman JS, Gordon R, Kristoff B, Fuentes F. Changes in cognitive andlanguage functioning of preschool children with autism. J Autism Dev Disord 1991Sep;21(3):281-90.Harris SL, Handleman JS. Preschool education programs for children with autism. Austin,(TX): Pro-Ed; 1994.Hibbs ED, Jensen PS (ed.). Psychosocial treatments for child and adolescent disorders:empirically based strategies for clinical practice. Washington (DC): American PsychologicalAssociation; 1996.Howlin P, Rutter M. Mothers' speech to autistic children: a preliminary causal analysis.J Child Psychol Psychiatry 1989 Nov;30(6):819-43.Howlin P. Practitioner review: psychological and educational treatments for autism. J ChildPsychol Psychiatry 1998 Mar;39(3):307-22.Howlin P. Prognosis in autism: do specialist treatments affect long-term outcome?  Eur ChildAdolesc Psychiatry 1997;6(2):55-72.Howlin P. When is a significant change not significant?  J Autism Dev Disord 1997Jun;27(3):347-8.Huebner RA, Emery LJ. Social psychological analysis of facilitated communication:implications for education. Ment Retard 1998 Aug;36(4):259-68.Hurth J, Shaw E, Izeman SG, Whaley K, Rogers SJ. Areas of agreement about effectivepractices among programs serving young children with autism spectrum disorders.  InfantsYoung Children 1999 Oct;12(2):17-26.Is Lovaas the only game in town: intervention programs for autistic children.  In: Cohen S,editor. Targeting autism: what we know, don't know and can do to help young children withautism and related disorders. Berkeley, (CA): University of California Press; 1998. p84-118.BC Office of Health Technology AssessmentAutism and Lovaas treatment47Jacobson JW, Mulick JA, Green G. Cost-benefit estimates for early intensive behavioralintervention for young children with autism: general model and single state case. BehavInterventions 1998;13(4):201-26.Jacobson JW, Mulick JA, Green G. Financial cost and benefits of intensive early interventionfor young children with autism: Pennsylvania model achieving cost savings. Schenectady(NY): Independent Living in the Capital District; 1996.Jocelyn LJ, Casiro OG, Beattie D, Bow J, Kneisz J. Treatment of children with autism: arandomized controlled trial to evaluate a caregiver-based intervention program in communityday-care centers. J Dev Behav Pediatr 1998 Oct;19(5):326-34.Kanner L. Follow-up study of eleven autistic children originally reported in 1943. Focus onAutistic Behavior 1992;7(5):1-11.Kaplan CA, Hussain S. Use of drugs in child and adolescent psychiatry. Br J Psychiatry1995;166:291-98.Kaplan M, Carmody DP, Gaydos A. Postural orientation modifications in autism in responseto ambient lenses. Child Psychiatr Human Dev 1996 Win;27(2):81-91.Kazdin AE. Combined and multimodal treatments in child and adolescent psychotherapy:issues, challenges, and research directions. Clin Psychol Sci Prac 1996;3:69-100.Kazdin AE, Weisz JR. Identifying and developing empirically supported child andadolescent treatments. J Consult Clin Psychol 1998;66(1):19-36.Kazdin AE. Replication and extension of behavioral treatment of austistic disorder. Am JMent Retard 1993;97(4):377-9.Klorman R. Psycholphysiological determinants. In: Hersen M, Ammerman RT, editors.Advanced abnormal child psychology. Hillsdale (NJ): Lawrence Erlbaum Associates Inc;1995. p59-85.Kobayashi R, Yoshinaga K. A follow-up study of 201 children with autism in Kyushu andYamaguchi areas, Japan. J Autism Dev Disord 1992;22(3):395-411.Koegel RL, Bimbela A, Schreibman L. Collateral effects of parent training on familyinteractions. J Autism Dev Disord 1996 Jun;26(3):347-59.Koegel RL, Koegel LK. Extended reductions in stereotypic behavior of students with autismthrough a self-management treatment package. J Appl Behav Anal 1990 Spr;23(1):119-27.[L'autisme]: Autism. Paris, France: Agence Nationale pour le Developpement de l'EvaluationMedicale; 1994.Larsen FW, Mouridsen SE. The outcome in children with childhood autism and Aspergersyndrome originally diagnosed as psychotic: a 30-year follow-up study of subjectshospitalized as children. Eur Child Adolesc Psychiatry 1997 Dec;6(4):181-90.BC Office of Health Technology AssessmentAutism and Lovaas treatment48LeBlanc R. Educational management of self-injurious behavior. Acta Paedopsychiatr1993;56(2):91-8.Lewis MH. Brief report: psychopharmacology of autism spectrum disorders. J Autism DevDisord 1996 Apr;26(2):231-5.Lonigan CJ, Elbert JC, Johnson SB. Empirically supported psychosocial interventions forchildren: an overview. J Clin Child Psychol 1998;27(2):138-45.Loose VW. Guidelines for benefit-cost analysis. Victoria (BC): Environment and Land UseCommittee Secretariat; 1977.Lord C. Follow-up of two-year-olds referred for possible autism. J Child Psychol Psychiatry1995;36(8):1365-82.Lord C, Schopler E. The role of age at assessment, developmental level, and test in thestability of intelligence scores in young autistic children. J Autism Dev Disord1989;19(4):483-99.Lovaas OI. Behavioral treatment and normal educational and intellectual functioning inyoung autistic children. J Consult Clin Psychol 1987;55(1):3-9.Lovaas OI, Smith T, McEachin JJ. Clarifying comments on the young autism study: reply toSchopler, Short, and Mesibov. J Consult Clin Psychol 1989;57(1):165-7.Lovaas OI, Smith T. A comprehensive behavioral theory of autistic children : paradigm forresearch and treatment. J Behav Ther  Exp Psychiatr 1989;20(1):17-29.Lovaas OI. Concerns about misinterpretation and placement of blame. Am Psychol 1989Sep;44(9):1243-4.Lovaas OI. The development of a treatment-research project for developmentally disabledand autistic children. J Appl Behav Anal 1993;26(4):617-30.Lovaas OI, Buch G. Editors' introduction. Res Dev Dis 1992;13:1-7.Lovaas OI. Intensive and early behavioral intervention with autism. Geneva CenterSymposium on Autism 1996 Conference Proceedings. Geneva Center, 1996.Lovaas OI, Smith T. Intensive and long-term treatments for clients with destructivebehaviors. In: Thompson T, Gray DB. editors. Destructive behavior in developmentaldisabilities: diagnosis and treatment. London: Sage Publications; 1994. p.243-60.Lovaas OI, Buch G. Intensive behavioral intervention with young children with autism. In:Singh NN, editor.  Prevention and treatment of severe behavior problems: models andmethods in developmental disabilities.  Pacific Grove (CA): Brooks / Cole Publishing; 1997.p62-85.BC Office of Health Technology AssessmentAutism and Lovaas treatment49Lovaas OI, Smith T. Intensive behavioral treatment for young autistic children. In: LaheyBB, Kazdin AE, editors.  Advances in clinical child psychology. New York: Plenum Press;1988.  p285-324.Lovaas OI, Calouri K, Jada J. The nature of behavioral treatment and research with youngautistic persons. In: Gilberg C, editor. Diagnosis and treatment of autism. New York: PlenumPress; 1989. p285-305.Lovaas OI. Parents as therapists.  In: Rutter M, Schopler E, editors. Autism: a reappraisal ofconcepts and treatment New York: Plenum Press; 1978. p369-378.Lovaas OI, Favell JE. Protection for clients undergoing aversive/restrictive interventions.Education and Treatment of Children 1987;10(4):311-25.Lovaas OI, Ackerman A, Alexander D, Firestone P, Perkins M, Young DB, Carr EG,Newsom C. Teaching developmentally disabled children: the ME book. Baltimore (MD):University Park Press; 1981.Lovaas OI, Smith T. There is more to operant theory and practice: comment. Am J MentRetard 1991;96(3):324-7.Luiselli JK, Wolongevicz J, Egan P, Amirault D, Sciaraffa N, Treml T. The Family SupportProgram: description of a preventive, community- based behavioral intervention for childrenwith pervasive developmental disorders. Child Fam Behav Ther 1999;21(1):1-18.Mahoney G, Kaiser A, Girolametto L, MacDonald J, Robinson C, Safford P, Spiker D.Parent education in early intervention: a call for a renewed focus. Top Early Child Spec Educ1999;19(3):131-40.Matson JL, Benavidez DA, Compton LS, Paclawskyj T, Baglio C. Behavioral treatment ofautistic persons: a review of research from 1980 to the present. Res Dev Dis 1996 Nov-Dec;17(6):433-65.Maurice C, Green G, Luce SC. Behavioral intervention for young children with autism: amanual for parents and professionals. Austin (TX): PRO-ED Inc.; 1996.McClannahan LE, Krantz PJ. Princeton child development institute. Behavior and SocialIssues 1997;7(1):65-8.McEachin JJ, Smith T, Lovaas OI. Long-term outcome for children with autism whoreceived early intensive behavioral treatment. Am J Ment Retard 1993;97(4):359-72.McEachin JJ. Outcome of autistic children receiving intensive behavioral treatment:psychological status 3 to 12 years later. Los Angeles (CA): University of California; 1987.Meisels SJ, Shonkoff JP. Handbook of early childhood intervention. New York (NY):Cambridge University Press; 1990.BC Office of Health Technology AssessmentAutism and Lovaas treatment50Mesibov GB. A comprehensive program for serving people with autism and their families:the TEACCH model. In: Matson JL, editor. Autism in children and adult: etiology,assessment, and intervention. Pacific Grove,(CA): Brooks/Cole Publishing Co.;1994. p85-97.Mesibov GB. Treatment outcome is encouraging. Am J Ment Retard 1993 Jan;97(4):379-80.Mulick JA. Making a difference: comments on Weiss. Behav Interventions 1999;14:29-34.Mundy P, Crowson M. Joint attention and early social communication: implications forresearch on intervention with autism. J Autism Dev Disord 1997;27(6):653-76.Mundy P. Normal versus high-functioning status in children with autism. Am J Ment Retard1993 Jan;97(4):381-4.Newsom C. Autistic disorder. In: Mash EJ, Barkley RA, editors. Treatment of childhooddisorders. 2nd ed. New York: The Guilford Press; 1998. p416-67.Nickel RE. Controversial therapies for young children with developmental disabilities.Infants Young Children 1996;8(4):29-40.Nordin V, Gillberg C. The long-term course of autistic disorders: update on follow-upstudies. Acta Psychiatr Scand 1998;97(2):99-108.Olley JG, Robbins FR, Morelli-Robbins M. Current practices in early intervention forchildren with autism. In: Schopler E, Mesibov GB. editors. Preschool issues in autism.Current issues in autism. New York, (NY): Plenum Press; 1993. p223-45.Ozonoff S, Cathcart K. Effectiveness of a home program intervention for young childrenwith autism. J Autism Dev Disord 1998 Feb;28(1):25-32.Perry R, Cohen I, DeCarlo R, Hertzig M. Applied behavioral analysis: astonishing results?:reply. J Am Acad Child Adolesc Psychiatry 1995 Oct;34(10):1256.Perry R, Cohen I, DeCarlo R. Case study: deterioration, autism, and recovery in two siblings.J Am Acad Child Adolesc Psychiatry 1995 Feb;34(2):232-7.Pfeiffer SI, Nelson DD. The cutting edge in services for people with autism. J Autism DevDisord 1992;22(1):95-105.Pisula E. [Early detection of autism in children: review of literature]. Psychiatr Pol 1997 Jul-Aug;31(4):389-96.Pollard NL. Development of social interaction skills in preschool children with autism: areview of the literature. Child Fam Behav Ther 1998;20(2):1-16.Practice parameters for the assessment and treatment of children, adolescents, and adultswith autism and other pervasive developmental disorders. J Am Acad Child AdolescPsychiatry 1999;38(12 sup):32S-54S.Preliminary report: cost-benefit analysis of Lovaas treatment for autism and autism spectrumdisorder (ASD). Vancouver (BC): Columbia Pacific Consulting; 1999.BC Office of Health Technology AssessmentAutism and Lovaas treatment51Prizant BM, Wetherby AM. Understanding the continuum of discrete-trial traditionalbehavioral to social-pragmatic developmental approaches in communication enhancementfor young children with autism/PDD. Sem Speech Language 1998;19(4):329-53, 424.Riddle MA, Bernstein GA, Cook EH, Leonard HL, March JS, Swanson JM. Anxiolytics,adrenergic agents, and naltrexone. J Am Acad Child Adolesc Psychiatry 1999May;38(5):546-56.Rimland B. Critique of "Efficacy of vitamin B6 and magnesium in the treatment of autism".J Autism Dev Disord 1998 Dec;28(6):580-1.Rimland B, Baker SM. Brief report: alternative approaches to the development of effectivetreatments for autism. J Autism Dev Disord 1996 Apr;26(2):237-41.Rimland B, Edelson SM. Brief report: a pilot study of auditory integration training in autism.J Autism Dev Disord 1995;25(1):61-70.Rimland B, Edelson SM. Response to Howlin on the value of auditory integration training.J Autism Dev Disord 1998;28(2):169-70.Ritvo RZ, Papilsky SB. Effectiveness of psychotherapy. Current Opin Pediatr 1999;11(4):323-27.Rodier PM, Bryson SE, Welch JP. Minor malformations and physical measurements inautism: data from Nova Scotia. Teratology 1997;55(5):319-25.Rogers SJ. Brief report: early intervention in autism. J Autism Dev Disord 1996Apr;26(2):243-6.Rogers SJ. Empirically supported comprehensive treatments for young children with autism.J Clin Child Psychol 1998 Jun;27(2):168-79.Rogers SJ. Intervention for young children with autism: from research to practice. InfantsYoung Children 1999 Oct;12(2):1-16.Romanczyk RG. Comments on Weiss. Behav Interventions 1999;14:35-6.Roser K. A review of psychoanalytic theory and treatment of childhood autism.Psychoanalytic Rev 1996 Jun;83(3):325-41.Roth FP. Communicative intervention for children with psychiatric and communicationdisorders. Child Adolesc Psychiatr Clin North Am 1999 Jan;8(1):137.Rutter M. Autism research: prospects and priorities. J Autism Dev Dis 1996 Apr;26(2):257-75.Rutter M. Autistic children: infancy to adulthood. Semin Psychiatry 1970;2(4):435-50.Rutter M. The Emanuel Miller memorial lecture 1998: autism: two-way interplay betweenresearch and clinical work. J Child Psychol Psychiatry 1999;40( 2):169-88.BC Office of Health Technology AssessmentAutism and Lovaas treatment52Schopler E. Concerns about misinterpretation and uncritical acceptance of exaggeratedclaims. Am Psychol 1988 Aug;43(8):658.Schopler E, Short A, Mesibov G. Relation of behavioral treatment to "normal functioning":comment on Lovaas. J Consult Clin Psychol 1989;57(1):162-4.Schopler E, Mesibov GB, Hearsey K. Structured teaching in the TEACCH system. In:Schopler E, Mesibov GB, editors. Learning and cognition in autism. Current issues in autism.New York: Plenum Press; 1995. p243-68.Schopler E. Will your journal support parents advocating for intensive behavior therapy (theLovaas method) as an entitlement under Part H of the Individuals with Disabilities EducationAct?. J Autism Dev Disord 1998;28(1):91-2.Schreibman L. Brief report: the case for social and behavioral intervention. J Autism DevDisord 1996 Apr;26(2):247-50.Schroeder SR, LeBlanc JM, Mayo L. Brief report: a life-span perspective on the developmentof individuals with autism. J Autism Dev Disord 1996 Apr;26(2):251-5.Schumacker J, Schumacker C. What the Lovaas program means to parents. Autism B.C.Newsletter 1994; 2(3):5.Service planning guidelines autistic spectrum disorders. [Phoenix (AZ)]: ArizonaDepartment of Health Services; 1999.Shapiro T, Hertzig M. Applied behavioral analysis: astonishing results? J Am Acad ChildAdolesc Psychiatry 1995 Oct;34(10):1255-6.Sheinkopf SJ, Siegel B. Home-based behavioral treatment of young children with autism.J Autism Dev Disord 1998 Feb;28(1):15-23.Sherman J, Barker P, Lorimer P, Swinson R, Factor DC. Treatment of autistic children:relative effectiveness of residential, out-patient and home-based interventions. ChildPsychiatr Hum Dev 1988 Win;19(2):109-25.Sigman M. Behavioral research in childhood autism. In: Lezenweger MF, Haugaard JJ,editors. Frontiers of developmental psychopathology. New York: Oxford University Press;1996. p190-206.Sigman M, Arbelle S, Dissanayake C. Current research findings on childhood autism. Can JPsychiatry 1995;40(6):289-94.Smith T. Autism. In: Giles TR, editor. Handbook of effective psychotherapy.  New York(NY): Plenum Press; 1993.  p107-33.Smith T. Outcome of early intervention for children with autism. Clin Psychol Sci Pract 1999Spr;6(1):33-49.BC Office of Health Technology AssessmentAutism and Lovaas treatment53Smith T. Concerns about nonspecific factors in the treatment of developmental disabilities.Am Psychol 1988 Aug;43(8):657-8.Smith T, Eikeseth S, Klevstrand M, Lovaas OI. Intensive behavioral treatment forpreschoolers with severe mental retardation and pervasive developmental disorder. Am JMent Retard 1997 Nov;102(3):238-49.Smith T, Lovaas OI. Seminar on applied behavior analysis treatment for autism: intensiveand early behavioral intervention with autism: the UCLA Young Autism Project. InfantsYoung Children 1998;10(3):67-78.Smith T, Lovaas OI. Sex and bias: reply to Boyd. J Autism Dev Disord 1998 Aug;28(4):343-44.Smith T, Lovaas OI. The UCLA young autism project: a reply to Gresham and MacMillan.Behav Disord 1997;22(4):202-18.Smith T, McEachin JJ, Lovaas OI. Comments on replication and evaluation of outcome. AmJ Ment Retard 1993;97(4):385-91.Smith T, Parker T, Taubman M, Lovaas OI. Transfer of staff training from workshops togroup homes: a failure to generalize across settings. Res Dev Disord 1992;13(1):57-71.Sprague RL. Ethics of treatment evaluation: balancing efficacy against other considerations.In: Thompson T, Gray DB, editors. Destructive behavior in developmental disabilities:diagnosis and treatment. London: Sage Publications; 1994. p.293-311.Stahmer AC. Teaching symbolic play skills to children with autism using pivotal responsetraining. J Autism Dev Disord 1995 Apr;25(2):123-41.Stone WL, Lee EB, Ashford L, Brissie J, Hepburn SL, Coonrod EE, Weiss BH. Can autismbe diagnosed accurately in children under 3 years?. J Child Psychol Psychiatry1999;40(2):219-26.Stone WL, Ousley OY, Yoder PJ, Hogan KL, Hepburn SL. Nonverbal communication intwo- and three-year-old children with autism. J Autism Dev Disord 1997 Dec;27(6):677-96.Strain PS. Comprehensive evaluation of intervention for young autistic children. Top EarlyChild Spec Educ 1987 Sum;7(2):97-110.Strickland D, Marcus LM, Mesibov GB, Hogan K. Brief report: two case studies usingvirtual reality as a learning tool for autistic children. J Autism Dev Disord 1996Dec;26(6):651-9.Szatmari P, Streiner DL. The effect of selection criteria on outcome studies of childrenwith pervasive developmental disorders (PDD). Eur Child Adolesc Psychiatry 1996Dec;5(4):179-84.Szot Z. The method of stimulated serial repetitions of gymnastic exercises in therapy ofautistic children. J Autism Dev Disord 1997 Jun;27(3):341-2.BC Office of Health Technology AssessmentAutism and Lovaas treatment54Tallal P, Merzenich M, Miller S, Jenkins W. Language learning impairment: integratingresearch and remediation. Scand J Psychol 1998 Sep;39(3):197-9.Thomsen PH. Obsessive-compulsive disorder in children and adolescents: clinicalguidelines. Eur Child Adolesc Psychiatry 1998;7(1):1-11.Thyer BA. Effective psychosocial treatments for children: a selected review. Early Child DevCare 1995;106:137-47.Thyer BA. Promoting an empiricist agenda within the human services: an ethical andhumanistic imperative. J Behav Ther Exp Psychiatry 1995;26(2):93-8.Tregear S, Turkelson C, Doggett D, Tappe K, Lyons C.  Comprehensive programs for thetreatment of children with autism [abstract].  16th Annual Meeting of the InternationalSociety of Technology Assessment in Health Care; 2000 Jun 18-21; The Hague, TheNetherlands.  Montreal: ISTAHC; 2000.Ulloa Estrada A, Pinsof WM. The effectiveness of family therapies for selected behavioraldisorders of childhood. J Marital Fam Ther 1995;21(4):403-40.Volkmar FR, editor. Autism and pervasive developmental disorders. Cambridge: CambridgeUniversity Press; 1998.Weiss MJ, Green G, Mulick JA, Romanczyk RG. Differential rates of skill acquisition andoutcomes of early intensive behavioral intervention for autism. Behav Interventions1999;14(1):3-36.Werry JS. Pervasive developmental, psychotic, and allied disorders. In: Hechtman LT, editor.Do they grow out of it?: long term outcomes of childhood disorders. Washington (DC):American Psychiatric Press; 1996. p. 195-223.Zimmerman AW, Jinnah HA, Lockhart PJ. Behavioral neuropharmacology. Ment RetardDev Disord Res Rev 1998;4( 1):26-35.Zuddas A, Ledda MG, Fratta A, Muglia P, Cianchetti C. Clinical effects of clozapine onautistic disorder. Am J Psychiatry 1996 May;153(5):738.BC Office of Health Technology AssessmentAutism and Lovaas treatment55REFERENCES                                                          1 Volkmar FR, editor. Autism and pervasive developmental disorders. Cambridge:Cambridge University Press; 1998.2 World Health Organization.  ICD-10: International statistical classification of diseasesand related health problems: Tenth revision. Geneva : World Health Organization; 1992-1994.3 Best L, Milne R, Wessex Institute for Health Research and Development. Auditoryintegration training in autism. Development and evaluation committee report; No. 66.Southampton, England: Wessex Institute for Health Research and Development, 1997.4 Bristol MM, Cohen DJ, Costello EJ, Denckla M, Eckberg TJ, Kallen R, Kraemer HC,Lord C, Maurer R, McIlvane WJ, Minshew N, Sigman M, Spence MA. State of thescience in autism: report to the National Institutes of Health. J Autism Dev Disord 1996Apr;26(2):121-54.5 Diagnostic and statistical manual of mental disorders: DSM-IV.  Washington (DC):American Psychiatric Association; 1994.6 Miller AR. Developmental Pediatrician. University of British Columbia. Vancouver(BC). (Personal communication, January 31, 2000).7 Howlin P. Prognosis in autism: do specialist treatments affect long-term outcome? EuChild Adolesc Psychiatry 1997;6(2):55-72.8 Nordin V, Gillberg C. The long-term course of autistic disorders: update on follow-upstudies. Acta Psychiatr Scand 1998;97(2):99-108.9 Jocelyn LJ, Casiro OG, Beattie D, Bow J, Kneisz J. Treatment of children with autism:a randomized controlled trial to evaluate a caregiver-based intervention program incommunity day-care centers. J Dev Behav Pediatr 1998 Oct;19(5):326-34.10 Campbell M, Cueva JE, Hallin A. Autism and pervasive developmental disorders. In:Wiener JM, editor. Diagnosis and psychopharmacology of childhood and adolescentdisorders. 2nd ed. New York: John Wiley & Sons; 1996. p. 151-92.11 Campbell M, Cueva JE. Psychopharmacology in child and adolescent psychiatry:a review of the past seven years: Part I. J Am Acad Child Adolesc Psychiatry 1995Sep;34(9):1124-32.12 Campbell M, Schopler E, Cueva JE, Hallin A. Treatment of autistic disorder. J Am AcadChild Adolesc Psychiatry 1996 Feb;35(2):134-43.13 Baer DM, Wolf MM, Risley TR.  Some current dimensions of applied behavioralanalysis. J Appl Behav Anal 1968;1:91-7.14 Rogers SJ. Brief report: early intervention in autism. J Autism Dev Disord 1996Apr;26(2):243-6.BC Office of Health Technology AssessmentAutism and Lovaas treatment56                                                                                                                                                                            15 Matson JL, Benavidez DA, Compton LS, Paclawskyj T, Baglio C. Behavioral treatmentof autistic persons: a review of research from 1980 to the present. Res Dev Dis 1996Nov-Dec;17(6):433-65.16 British Columbia. Ministry for Children and Families, British Columbia.  Ministry ofEducation. Autism action plan. Victoria (BC): Ministry for Children and Families, 1999.17 British Columbia. Ministry for Children and Families, British Columbia.  Ministry ofEducation. Autism action plan: implementation plan. Victoria (BC): Ministry forChildren and Families, 1999. p318 Lonigan CJ, Elbert JC, Johnson SB. Empirically supported sychosocial interventions forchildren: an overview. J Clin Child Psychol 1998;27(2):138-4.19 Gresham FM, MacMillan DL. Autistic recovery?: an analysis and critique of the empiricalevidence on the early intervention project. Behav Disord 1997 Aug;22(4):185-201.20 Sackett DL.  Clinical epidemiology: a basic science for clinical medicine.  2nd ed.Boston: Little, Brown; 1991.21 Lovaas OI. Behavioral treatment and normal educational and intellectual functioning inyoung austic children. J Consult and Cli  Psych 1987;55(1):3-9.22 McEachin JJ, Smith T, Lovaas OI. Long-term outcome for children with autism whoreceived early intensive behavioral treatment. Am J Ment Retard 1993;97(4):359-72.23 Birnbrauer JS, Leach DJ.  The Murdoch early intervention programme after 2 years.Behav Change 1993;10(2):63-74.24 Sheinkopf SJ, Siegel B. Home-based behavioral treatment of young children with autism.J Autism Dev Disord 1998 Feb;28(1):15-23.25 Ozonoff S, Cathcart K. Effectiveness of a home program intervention for young childrenwith autism. J Autism Dev Disord 1998 Feb;28(1):25-32.26 Rogers SJ. Empirically supported comprehensive treatments for young children withautism. J Clin Child Psychol 1998 Jun;27(2):168-79.27 Tregear S, Turkelson C, Doggett D, Tappe K, Lyons C.  Comprehensive programs for thetreatment of children with autism [abstract].  16th Annual Meeting of the InternationalSociety of Technology Assessment in Health Care; 2000 Jun 18-21; The Hague, TheNetherlands.  Montreal: ISTAHC; 2000.28 Green G. Early behavioral intervention for autism: What does research tell us?  In:Maurice C, Green G, Luce SC, editors. Behavioral intervention for young children withautism: a manual for parents and professionals. Austin (TX): PRO-ED Inc; 1996. p29-44.29 Smith T, Lovaas OI. Seminar on applied behavior analysis treatment for autism:intensive and early behavioral intervention with autism: the UCLA Young AutismProject. Infants Young Children 1998;10(3):67-78.BC Office of Health Technology AssessmentAutism and Lovaas treatment57                                                                                                                                                                            30 Schopler E, Short A, Mesibov G. Relation of behavioral treatment to “normal functioning”:comment on Lovaas. J Consult Clin Psych 1989;57(1):162-4.31 Lovaas OI, Smith T, McEachin JJ. Clarifying comments on the young autism study: replyto Schopler, Short, and Mesibov.  J Consult Clin Psychology 1989;57(1):165-7.32 Boyd RD. Sex as a possible source of group inequivalence in Lovaas (1987)  J AutismDev Dis 1998 Jun;28(3):211-5.33 Smith T, Lovaas OI. Sex and bias: reply to Boyd. J Autism Dev Disord 1998Aug;28(4):343-44.34 Boyd RD. Sex and bias: reply to Smith and Lovaas. J Autism Dev Dis 1998;28(4):344.35 Smith T, Lovaas OI. The UCLA young autism project: a reply to Gresham andMacMillan. Behav Disord 1997;22(4):202-18.36 Gresham FM, MacMillan DL. Denial and defensiveness in the place of fact and reason:rejoinder to Smith and Lovaas. Behav Disord 1997;22(4):219-30.37 Gresham FM, MacMillan DL. Early Intervention Project: can its claims be substantiatedand its effects replicated? J Autism Dev Disord 1998 Feb;28(1):5-13.38 Schopler E. Will your journal support parents advocating for intensive behavior therapy(the Lovaas method) as an entitlement under Part H of the Individuals with DisabilitiesEducation Act?  J Autism Dev Disord 1998;28(1):91-2.39 Mesibov GB. Treatment outcome is encouraging. Am J Ment Retard 1993 Jan;97(4):379-80.40 Mundy P. Normal versus high-functioning status in children with autism. Am J MentRetard 1993 Jan;97(4):381-4.41 Kazdin AE. Replication and extension of behavioral treatment of autistic disorder. Am JMent Retard 1993;97(4):377-9.42 Baer DM. Commentaries on McEachin, Smith, and Lovaas: quasi-random assignmentcan be as convincing as random assignment. Am J Ment Retard 1993 Jan;97(4):373-5.43 Foxx RM. Sapid effects awaiting independent replication.  Am J Ment Retard 1993Jan;97(4):375-6.44 Smith T, McEachin JJ, Lovaas OI. Comments on replication and evaluation of outcome.Am J Ment Retard 1993;97(4):385-91.45 Connor M. A review of behavioural early intervention programmes for children withautism. Educ Psychol Pract 1998;14(2):109-17.46 Lovaas OI, Ackerman A, Alexander D, Firestone P, erkins M, Young DB, Carr EG,Newsom C. Teaching developmentally disabled children: the ME book. Baltimore (MD):University Park Press; 1981.BC Office of Health Technology AssessmentAutism and Lovaas treatment58                                                                                                                                                                            47 Lovaas OI. Parents as therapists.  In: Rutter M, Schopler E, editors. Autism: Areappraisal of concepts and treatment New York: Plenum Press 1978. p. 369-378.48 Smith T. Outcome of early intervention for children with autism. Clin Psychol Sci Pract1999 Spr;6(1):33-49.49 Smith T. Are other treatments effective? In: Maurice C, Green G, Luce SC, editors.Behavioral intervention for young children with autism: a manual for parents andprofessionals. Austin (TX): PRO-ED Inc; 1996. p25-59.50 Lord C. Follow-up of two-year-olds referred for possible autism. J Child P ycholPsychiatry 1995;36(8):1365-82.51 Cox A, Charman T, Baron-Cohen S, Drew A, Klein K, Baird G, Swettenham J,Wheelwright S. Autism spectrum disorders at 20 and 42 months of age: stability ofclinical and ADI-R diagnosis. J Child Psychol Psychiatry 1999;40(5):719-32.52 Stone WL, Lee EB, Ashford L, Brissie J, Hepburn SL, Coonrod EE, Weiss BH. Canautism be diagnosed accurately in children under 3 years?. J Child Psychol Psychiatry1999;40(2):219-26.53 Rutter M. Autism research: prospects and priorities. J Autism Dev Dis 1996Apr;26(2):257-75.54 Anderson SR, Avery DL, i Pietro EK, Edwards GL, Christian WP. Intensive home-based early intervention with autistic children. Educ Treat Children 1987;10(4):352-66.55 Harris SL, Handleman JS, Gordon R, Kristoff B, Fuentes F. Changes in cognitive andlanguage functioning of preschool children with autism. J Autism DevDisord 1991Sep;21(3):281-90.56 Fenske EC, Zalenski S, Krantz PJ, McClannahan LE. Age at intervention and treatmentoutcome for autistic children in a comprehensive intervention program. AnalysisIntervention Dev Disord 1985;5:49-58.57 Clinical practice guideline: Report of the recommendations. Autism/pervasivedevelopmental disorders, assessment and intervention for young children (age 0-3 years).Albany (NY): New York State Department of Health; 1999.58 Field MJ, Lohr KN. Editors. Guidelines for Clinical Practice: from Development to Use.Committee on Clinical Practice Guidelines, Institute of Medicine, Division of HealthSciences Policy, Division of Health Promotion and Disease Prevention. Washington, DC:National Academy Press, 1992.59 Dawson G, Osterling J. Early intervention in autism. In: Gural ick MJ, editor.  Theeffectiveness of early intervention.  Baltimore (MD): P.H. Brookes; 1997.  p307-26.60 Hurth J, Shaw E, Izeman SG, Whaley K, Rogers SJ. Areas of agreement about effectivepractices among programs serving young children with autism spectrum disorders.Infants Young Children 1999 Oct;12(2):17-26.

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.52383.1-0045257/manifest

Comment

Related Items