Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

The transition of British Columbia’s tertiary psychiatric services from hospital to community : consideration… Petersen, Karen Leanne 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2015_september_petersen_karen.pdf [ 816.94kB ]
Metadata
JSON: 24-1.0166451.json
JSON-LD: 24-1.0166451-ld.json
RDF/XML (Pretty): 24-1.0166451-rdf.xml
RDF/JSON: 24-1.0166451-rdf.json
Turtle: 24-1.0166451-turtle.txt
N-Triples: 24-1.0166451-rdf-ntriples.txt
Original Record: 24-1.0166451-source.json
Full Text
24-1.0166451-fulltext.txt
Citation
24-1.0166451.ris

Full Text

The Transition of British Columbia’s Tertiary Psychiatric Services from Hospital to Community:  Consideration of Clinical, Psychosocial, and Transinstitutionalization Outcomes   by Karen Leanne Petersen  B.A., The University of British Columbia, 1997 B.Ed., The University of British Columbia, 2002  A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF  Doctor of Philosophy in THE FACULTY OF GRADUATE AND POSTDOCTORAL STUDIES (Experimental Medicine)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver)  July 2015 © Karen Leanne Petersen, 2015 ii  Abstract Deinstitutionalization of severely and persistently mentally ill individuals has been one of the most significant developments in mental health policy and practice in the last 70 years.  This dissertation is comprised of four interconnected papers which together examine the redevelopment of tertiary psychiatric services in British Columbia (BC).  The first paper reports on a cohort of patients who moved from Riverview Hospital (RVH), BC’s only large-scale tertiary psychiatric hospital, to community-based facilities in the Vancouver Island, Interior, Northern, and Fraser Health Authorities between 2001 and 2004.  The variety and characteristics of the facilities participants moved to over the course of the RVH closure and changes in clinical and psychosocial characteristics of participants over a 5-year period are described.  The next paper considers the psychosocial, clinical, and risk profile of the final cohort of RVH patients and compares and contrasts those characteristics with a forensic sample.  In addition, the prevalence of a variety of negative outcomes (i.e., aggression, inappropriate sexual behaviour, self-harm, suicide, self-neglect, unauthorized leave, substance abuse, victimization, and stalking) in a six month time frame are compared.   The third paper is a validation of the Short-Term Assessment of Risk and Treatability (START) in forensic mental health practice.  It addresses three specific research questions:  (1) What is the clinical and risk profile of a large Canadian forensic psychiatric cohort?  (2) What are the psychometric properties of the START in a forensic service when the evaluations are completed by clinical teams?  (3) How are these profiles and START outcomes similar or different among male and female patients and across various settings (i.e., hospital versus clinic) or security levels (i.e., maximum, medium, minimum)?  The fourth paper compares the psychometric properties of START in a forensic and civil psychiatric sample.   iii  The results from this dissertation provide evidence that a well-planned and well-resourced closure of a civil psychiatric hospital do not lead to negative outcomes such as transinstitutionalization and homelessness and suggest that START is a potentially useful tool for psychiatric patients.      iv  Preface Chapters 2-5 each consist of a paper published or submitted for publication.  The contribution of each co-author and UBC Ethics numbers are outlined below.  Chapter 2:  Redevelopment of Tertiary Psychiatric Services in British Columbia:  A Prospective Study of Clinical, Social, and Residential Outcomes of Former Long-Stay Inpatients (UBC Ethics # H10-00497).  Petersen conducted the data analyses and wrote the first draft of the manuscript.  Lesage designed the study and wrote the protocol.  Groden contributed to the design and organization of the study and collected the data.  Schmitz contributed to study design and statistical analysis.  Stip provided design input and unique knowledge of the population and measures.  Goldner contributed to design and provided local expertise.  Arnold contributed to design and provided expertise in local mental health policy and practice.  Nicholls provided expertise in the area of deinstitutionalization research, Riverview Hospital Redevelopment in British Columbia and oversaw the development of the manuscript.  All authors contributed to and have approved the final manuscript.  This paper has been reproduced with permission from the journal.  Chapter 3:  Forensic and tertiary civil psychiatric inpatients: Comparing psychosocial, clinical, and risk profiles, and the prevalence of negative outcomes (UBC Ethics #H06-80881).  Petersen conducted the data analyses, wrote the first draft of the manuscript, contributed to the study design, collected data and supported the execution of the study as research coordinator (i.e. training RAs, overseeing data collection, etc.).  Nicholls oversaw the study design, directed the research and the development of the manuscript.  Brink provided forensic expertise as well as v  contributing to the forensic arm of the study.  Douglas contributed the study design especially in the area of measure selection.  Gagnon contributed to the study design and provided statistical expertise.  Schmitz contributed statistical expertise as well as a larger health prospective in study design.  Leech contributed to the study design and implementation as well as data cleaning and analysis.  Lesage provided expertise in the overlap between forensic and civil psychiatric patients.  All authors contributed to and have approved the final manuscript.  Chapter 4:  A Clinical and Risk Profile of Forensic Psychiatric Patients:  Treatment Team STARTs in a Canadian Service (UBC Ethics #H09-00097).  Nicholls:  wrote the first draft of the manuscript and conducted the data analyses.  Petersen:  contributed to the writing of the manuscript and conducted some of the data analysis.  Brink:  provided forensic expertise in the interpretation of the data.  Webster:  provided risk assessment expertise.  All authors contributed to and have approved the final manuscript.  This paper has been reproduced with permission from the journal.  Chapter 5:  The psychometric properties of START: Comparing and contrasting a forensic and a civil psychiatric inpatient cohort.  Petersen conducted the data analyses, wrote the first draft of the manuscript, contributed to the study design, collected data and supported the execution of the study as research coordinator (i.e. training RAs, overseeing data collection, etc.).  Nicholls contributed to the study design, oversaw the research and the development of the manuscript.  Brink provided forensic expertise as well as contributing to the forensic arm of the study.  Douglas contributed the study design especially in the area of measure selection.  Gagnon contributed to the study design and provided statistical expertise.  Schmitz contributed statistical vi  expertise as well as a larger health prospective in study design.  Leech contributed to the study design and implementation as well as data cleaning and analysis.  Lesage provided expertise in the overlap between forensic and civil psychiatric patients.  All authors contributed to and have approved the final manuscript.  vii  Table of Contents  Abstract .......................................................................................................................................... ii Preface ........................................................................................................................................... iv Table of Contents ........................................................................................................................ vii List of Tables ............................................................................................................................... xii List of Figures ............................................................................................................................. xiv List of Abbreviations ...................................................................................................................xv Acknowledgements .................................................................................................................. xviii Dedication ................................................................................................................................... xix Chapter 1: Introduction ................................................................................................................1 1.1 Deinstitutionalization .................................................................................................... 2 1.2 Is Deinstitutionalization Leading to Transinstitutionalization? ............................... 5 1.3 Patient Safety, Risk Assessment, and Risk Management .......................................... 7 1.4 Summary ...................................................................................................................... 11 Chapter 2: Redevelopment of Tertiary Psychiatric Services in British Columbia:  A Prospective Study of Clinical, Social, and Residential Outcomes of Former Long-Stay Inpatients ......................................................................................................................................14 2.1 Redevelopment of Psychiatric Services in British Columbia:  Prospective Study of Clinical, Social and Residential Outcomes ........................................................................... 14 2.2 Method ......................................................................................................................... 18 2.2.1 Design ....................................................................................................................... 18 2.2.2 Procedures ................................................................................................................. 18 viii  2.2.3 Measures ................................................................................................................... 19 2.2.4 Settings ...................................................................................................................... 21 2.2.5 Analysis..................................................................................................................... 22 2.3 Results .......................................................................................................................... 22 2.3.1 Sample....................................................................................................................... 22 2.3.2 Clinical and Psycho-Social Results .......................................................................... 24 2.3.3 Facility Characteristics.............................................................................................. 28 2.4 Discussion..................................................................................................................... 33 Chapter 3: Forensic and Tertiary Civil Psychiatric Inpatients: Comparing Psychosocial, Clinical, and Risk Profiles, and the Prevalence of Negative Outcomes ..................................36 3.1 Introduction ................................................................................................................. 36 3.2 Method ......................................................................................................................... 37 3.2.1 Setting ....................................................................................................................... 37 3.2.2 Sample....................................................................................................................... 38 3.2.3 Measures ................................................................................................................... 39 3.2.4 Procedures ................................................................................................................. 40 3.2.5 Analysis..................................................................................................................... 40 3.3 Results .......................................................................................................................... 41 3.3.1 Psychosocial Characteristics ..................................................................................... 41 3.3.2 Historical Risk Factors .............................................................................................. 43 3.3.3 START Risk Profile .................................................................................................. 44 3.3.4 Prevalence of Adverse Outcomes Six Months Following the Completion of START Assessments .......................................................................................................................... 46 ix  3.4 Discussion..................................................................................................................... 49 3.5 Conclusions .................................................................................................................. 51 Chapter 4: A Clinical and Risk Profile of Forensic Psychiatric Patients:  Treatment Team STARTs in a Canadian Service ..................................................................................................53 4.1 Introduction ................................................................................................................. 53 4.1.1 Short-Term Assessment of Risk and Treatability (START)..................................... 54 4.1.2 International Uptake of the START .......................................................................... 55 4.1.3 Research on the START ........................................................................................... 55 4.1.3.1 User Satisfaction ............................................................................................... 55 4.1.3.2 Implementation Findings .................................................................................. 56 4.1.3.3 Validation Studies ............................................................................................. 57 4.1.4 The Present Study ..................................................................................................... 59 4.2 Method ......................................................................................................................... 60 4.2.1 Sample....................................................................................................................... 60 4.2.2 Clinical and Risk Profile of the Sample .................................................................... 60 4.2.3 Setting ....................................................................................................................... 62 4.2.3.1 Implementation of the START in the BC FPSC ............................................... 62 4.2.4 Procedures ................................................................................................................. 64 4.2.5 Measures ................................................................................................................... 64 4.3 Results .......................................................................................................................... 65 4.3.1 START Reliability .................................................................................................... 65 4.3.2 Descriptive Statistics ................................................................................................. 65 4.3.3 START Across the Continuum of Care .................................................................... 67 x  4.3.4 Risk Estimates ........................................................................................................... 70 4.3.4.1 Co-Occurring Risks .......................................................................................... 72 4.3.4.2 Key Items and Critical Items ............................................................................ 73 4.3.4.3 Signature Risk Signs ......................................................................................... 75 4.3.4.4 THREAT Boxes (Threat of Harm that is Real Enactable Acute and Targeted 76 4.4 Discussion..................................................................................................................... 76 4.4.1 What is the Clinical and Risk Profile of a Large Canadian Forensic Psychiatric Cohort? .................................................................................................................................. 76 4.4.1.1 Profile as a Function of Gender ........................................................................ 77 4.4.1.2 Profile as a Function of Setting and Security Level ......................................... 78 4.4.2 What are the Psychometric Properties of the START in a Forensic Service when the Evaluations are Completed by Clinical Teams? ................................................................... 80 4.4.3 Limitations ................................................................................................................ 81 4.4.4 Conclusion ................................................................................................................ 81 Chapter 5: The Psychometric Properties of START:  Comparing and Contrasting a Forensic and a Civil Psychiatric Inpatient Cohort ...................................................................85 5.1 Introduction’ ............................................................................................................... 85 5.2 Method ......................................................................................................................... 87 5.2.1 Sample....................................................................................................................... 87 5.2.2 Setting ....................................................................................................................... 88 5.2.3 Procedures ................................................................................................................. 88 5.2.4 Measures ................................................................................................................... 89 5.2.5 Analysis..................................................................................................................... 90 xi  5.3 Results .......................................................................................................................... 91 5.3.1 Sample....................................................................................................................... 91 5.3.2 Inter-Rater Reliability ............................................................................................... 91 5.3.3 Base-Rates of Risk Outcomes ................................................................................... 92 5.3.4 Relationship between START Total Scores and the Adverse Outcomes ................. 93 5.3.5 Relationship between Low vs. Moderate/High START Risk Estimates and the Adverse Outcomes ................................................................................................................ 94 5.3.6 Incremental Validity of Risk Estimates over START Total Scores ......................... 95 5.1.1 ROC Analyses of START Total Scores and Risk Estimates .................................. 102 5.1.2 Two-Step Cluster Analysis ..................................................................................... 105 5.2 Discussion................................................................................................................... 105 Chapter 6: Discussion ................................................................................................................109 6.1 Realignment of Tertiary Psychiatric Services in British Columbia: Has this Province’s Redevelopment of Services Avoided the Negative Outcomes Historically Associated with Hospital Closures? .................................................................................... 109 6.2 Risk Profiles, Risk Assessment and Patient Safety in Forensic and Civil Psychiatric Patients ............................................................................................................... 110 6.3 Utility of START, in vivo, in a Forensic Setting..................................................... 112 6.4 Utility of START in a Civil and a Forensic Setting Considering Multiple Patient Safety Outcomes .................................................................................................................... 114 6.5 Methodological Considerations:  Strengths and Limitations ............................... 115 6.6 Summation and Future Directions .......................................................................... 117 References ...................................................................................................................................122 xii  List of Tables Table 2.1  Facility Characteristics   ................................................................................................ 21Table 2.2  Sociodemographic and Clinic Characteristics at Baseline   .......................................... 23Table 2.3  Clinical and Psychosocial Outcomes   ........................................................................... 26Table 2.4  Assessment Facilities   ................................................................................................... 29Table 3.1  Psychosocial Characteristics at Baseline   ..................................................................... 42Table 3.2  Historical Risk Factors at Baseline   .............................................................................. 43Table 3.3  START Risk Profile  ..................................................................................................... 45Table 3.4  Negative Outcomes:  Prevalence and Nature of Adverse Incidents in the Six Months Following the Completion of START Assessments   ..................................................................... 47Table 3.5  Prevalence of Severe Adverse Events in the Six Months Following the Completion of START Assessments   .................................................................................................................... 48Table 4.1  Characteristics of a Forensic Psychiatric Cohort   ......................................................... 62Table 4.2  START Scales by Gender   ............................................................................................ 66Table 4.3  Frequency of START Item Endorsement Among Forensic Inpatients  ........................ 66Table 4.4  START Scale Scores by Setting and Security Level   ................................................... 68Table 4.5  START Item Distributions by Hospital Security Level  ............................................... 69Table 4.6  Intercorrelations of START Risk Estimates as a Function of Client Gender   .............. 73Table 4.7  Frequency of START Items being Endorsed as Key Items or Critical Items   .............. 74Table 5.1  Summary of Relevant START Literature   .................................................................... 85Table 5.2  Socio/Demographic Characteristics at Baseline   .......................................................... 91Table 5.3  Base-Rates of Outcomes   .............................................................................................. 92xiii  Table 5.4  Mean Strength and Vulnerability Total Scores of Inpatients Who Experienced Adverse Outcomes   ........................................................................................................................ 93Table 5.5  Proportion of Civil and Forensic Psychiatric Patients with Low vs. Moderate/High START Risk Estimates and the Odds Ratios for the presence of Adverse Outcomes:  A six month follow-up  ............................................................................................................................ 95Table 5.6  Incremental Validity of Risk Estimates over START Total Scores   ............................ 96Table 5.7  Predicting Adverse Outcomes in Civil and Forensic Patients using START Assessments: ROC Analyses   ...................................................................................................... 103 xiv  List of Figures Figure 2.1  Assessment Facilities – Descriptive   ........................................................................... 32  xv  List of Abbreviations AUC – Area Under the Curve BA – Bachelor of Arts BC – British Columbia BPRS – Brief Psychiatric Rating Scale CAN – Canadian CANTAB – Cambridge Neuropsychological Test Automated Battery CI – Confidence Interval CPRF –Community Psychiatric Residential Facilities ESMS – European Services Mapping Schedule FPH – Forensic Psychiatric Hospital FPSC – Forensic Psychiatric Service Commission HCR-20 – Historical, Clinical, Risk Management-20 ICC – Intraclass Correlation Coefficient ILSS – Independent Living Skills Schedule iORNS – Inventory of Offender Risk,  Needs, and Strengths LOCS – Level of Care Survey L – Low MH – Moderate or High M – Mean MIC – Mean Inter-item Correlations MCITC – Mean Corrected Item-Total Correlation MOT – Motor Screening Task xvi  N - Number NCR – Not Criminally Responsible NCRMD – Not Criminally Responsible on account of Mental Disorder OAS – Overt Aggression Scale OAS-M - Overt Aggression Scale - Modified OR – Odds Ratio PAL – Paired Associates Learning PCL-SV – Psychopathy Checklist:  Screening Version RE – Risk Estimate ROC – Receiver Operating Characteristic RPI – Riverview Patient Inventory SAPRF – Structured Assessment of Protective Factors for Violence Risk SAVRY – Structured Assessment of Violence Risk in Youth SBS – Social Behaviour Schedule SD – Standard Deviation SPJ – Structured Professional Judgment SOS – Start Outcome Scale SRM – Spatial Recognition Memory SRO – Single Room Occupancy Hotel SPSS – Statistical Package for the Social Sciences START – Short-Term Assessment of Risk and Treatability TAPS –Team for the Assessment of Psychiatric Services  THREAT – Threat of Harm that is Real Enactable and Targeted xvii  TPRFs – Tertiary Psychiatric Residential Facilities  xviii  Acknowledgements  I offer my enduring gratitude to the faculty, staff, co-workers and my fellow students at UBC, FPH and SFU, who have inspired me to continue my work in this field. I owe particular thanks to Dr. Tonia Nicholls for her time, support, and mentorship.  I would also like to thank Drs. Lesage, Schmitz, and Krausz for bringing their diverse perspectives that resulted in deepening my knowledge.  I would like to thank all of the study participants who generously shared their personal experiences of moving out of Riverview Hospital into smaller community-based facilities.  Special thanks are owed to my parents, who have supported me throughout my years of education. xix  Dedication  I would like to dedicate my dissertation to all the family, friends, co-workers, and mentors who made its completion possible.   1  Chapter 1: Introduction Deinstitutionalization of severely and persistently mentally ill individuals has been one of the most significant developments in mental health policy and practice in the last 70 years (Micale, 2014).  A consequential number of large-scale tertiary psychiatric hospitals in North America and around the world have closed, are in the process of closing, or are being considered for closure (e.g., Leff, Trieman, Knapp & Hallam, 2000; Oshima, Mino, & Inomato, 2007; Alberta Health, 2015).  Despite the far-reaching ramifications of the deinstitutionalization movement, many questions remain regarding whether or not this dramatic shift in policy and practice will enhance mental health care from a fiscal, patient safety, or humanitarian perspective.  Opponents of this policy surmise that community-based care can lead to inadequate treatment, transinstitutionalization, increased family burden, and even homelessness (Bachrach, 2001; Lamb & Bacrach, 2001).  The extant literature also leaves many questions unanswered; there have been only a handful of prospective longitudinal research studies and results are mixed (e.g. Burns, 2014).  However, mental health services in the community are widely perceived to be the best method for delivering psychiatric services (WHO, 2001).  For example, the Canadian Senate produced a report, entitled Out of the Shadows at Last, which included 118 recommendations for transforming mental health care in Canada and highlighted the importance of delivering services in communities in which people live (Kirby, 2006).  Consistent with the international move toward community-base care, the Riverview Hospital Redevelopment Project was launched by British Columbia’s (BC’s) Ministry of Health Plan in 2002 to redevelop tertiary psychiatric care across the province.  It involved the closure of Riverview Hospital (RVH) and the realignment of psychiatric services into the community; generally this involved the opening of purpose-built or newly renovated facilities that were 2  intended to be smaller, more homelike, and closer to patients’ homes (BC Mental Health & Addiction Services, 2009).  At the time the research from this dissertation was initiated there were no studies of deinstitutionalization in a Canadian context, and there were limits to the generalizability of studies from around the world to the Canadian context.  This dissertation:  1) investigates deinstitutionalization of civil psychiatric care in BC; 2) examines the risks and needs of tertiary psychiatric patients; and 3) validates a widely used risk assessment measure that shows promise for risk assessment and the prevention and management of these patient safety concerns in both civil and forensic populations (START, Webster et al., 2004, 2009). 1.1 Deinstitutionalization In a study of 200 mental health professionals, deinstitutionalization was found to be one of the ten most important developments in mental health policy and practice in the last 70 years (Micale, 2014).  The ideals driving deinstitutionalization were laudable.  Practitioners wanted to provide the best possible care for their patients in the least restrictive environment.  Psychiatric hospitals in the not too distant past were uninspiring or even untherapeutic places (Fakhoury & Priebe, 2007).  The intention was to ensure that patients treated in the community could be closer to their support networks (e.g. family and friends) and remain active members of their communities.  There is a small body of research examining the closure of large-scale psychiatric hospitals and the transition to community-based care around the world.  Research results from early examples of deinstitutionalization (i.e., first generation deinstitutionalization:  1950s – 1990s) led many commentators to conclude that community-based care is inadequate and leads to negative outcomes such as homelessness, inadequate treatment, transinstitutionalization (i.e., patient care transferred to correctional facilities or forensic psychiatric facilities) and/or 3  premature death (Bachrach, 2001; Lamb & Bachrach, 2001).  However, the World Health Organization, in evaluating best practice for psychiatric care, asserted that the unfavourable results of first generation deinstitutionalization can be attributed to underfunding and poor management at the system level rather than limitations with community-based care models (2003).  Second generation deinstitutionalization research results from around the world are slowly being accumulated:  Italy (Barbato, 1998; Lesage & Tansella, 1993; D’Avanzo et al., 2003), the United States (e.g., Indiana - McGrew et al., 1999a, 1999b; Pescosolido et al., 1999), the United Kingdom (UK) (Leff et al., 2000; Leff & Trieman, 2000), Australia (Hobbs et al., 2000; Newton et al., 2000; Hobbs et al., 2002), Canada (e.g., Quebec - Trudel & Lesage, 2006; Lesage et al., 2000; Livingston, Nicholls, & Brink, 2011), and the Netherlands (Amsterdam) (Duurkoop & van Dyck, 2003).  Each of these studies has demonstrated that properly executed programs of deinstitutionalization are feasible and do not lead to negative clinical or social outcomes. Specifically, perhaps the most important longitudinal study in this field of research is the Team for the Assessment of Psychiatric Services (TAPS) project conducted in the UK (Leff et al., 2000).  With results that generally echo other studies, the TAPS study found improvements in domestic living skills, community living skills, and quality-of-life.  Patients also developed more friendships and enjoyed increased freedom after leaving the large-scale hospital and relocating to the smaller community-based facilities (Leff & Trieman, 2000).  The TAPS study also demonstrated no increase in mortality, homelessness, or criminal activity among its participants (Leff et al., 2000; Leff & Trieman, 2000; Leff et al., 1996; Dayson, 1993; Leff et al., 1994). However, there was no evidence of amelioration of participants’ clinical symptoms or undesirable social behaviours (Leff & Trieman, 2000).   Over and above clinical and socio-4  cultural evidence that community-based care is the best practice, there is also evidence that it is a cost-effective method of service delivery (Reinharz et al., 2000). Notwithstanding the positive results from the second generation of studies, there are still a substantial number of commentators in academia, policing, government, and the general public who question the utility of deinstitutionalization (e.g. Bachrach, 2001; Lamb & Bachrach, 2001).  In addition, differences between countries in terms of policy, implementation and measurement of outcomes has led to the belief there is still much to be learned about deinstitutionalization and community-based care, warranting further research (Priebe & Turner, 2003).  The differences between countries in health care provision and the ways in which deinstitutionalization has been implemented limit the generalizability of the prevailing research to the Canadian context.   In BC, tertiary psychiatric services have been undergoing reorganization and redevelopment since the 1960s (BC Government, 2012).  Although this process was far from linear and was punctuated by various delays and interruptions, Riverview Hospital (RVH), BC’s only tertiary care psychiatric hospital, underwent downsizing from approximately 4,000 beds in the early 1960s to approximately 500 beds in the early 1990s (Macfarlane et al., 1997; BC Mental Health & Addiction Services, 2010).  The last phase of the RVH Redevelopment Project, which began in 2002, was instigated by the release of a new Mental Health Plan for BC in 1998 (BC Ministry of Health, 1998).  The BC Mental Health Plan called for the continued redevelopment and decentralization of Riverview Hospital and the building of new purpose-designed, or renovated of small-, more home-like community-based facilities (BC Ministry of Health, 1998).  The provincial Ministry of Health committed CAN$138 million in capital funding to construct new facilities or renovate existing facilities where appropriate (BC Ministry of Health, 1998).  Additionally, CAN$4,050,000 of annualized funding was pledged (BC Mental 5  Health and Addiction Services, 2009).  This final phase of the RVH Redevelopment Project was completed in the summer of 2012 (BC Mental Health & Addiction Services, 2014).   1.2 Is Deinstitutionalization Leading to Transinstitutionalization? Researchers, politicians, policy makers and mental health activists have suggested that deinstutionalization has led to the criminalization of mental illness and the transinstitutionalization of severely mentally ill persons to other criminal justice and social agencies, particularly the forensic psychiatric system (e.g., Peternelj-Talyor, 2008; Priebe & Turner, 2003).  This perspective has drawn support from evidence of the steady increase in demand for forensic mental health services in many countries around the world (Jansman-Hart et al., 2011).  It is unknown, however, whether or not this increase is a result of changes in criminal or civil law, changes in the nature of the population (more refractory patients, more behaviourally difficult patients, more patients with comorbid substance use disorders), the failure of deinstitutionalized civil psychiatric systems to transfer resources and services to the community to treat the most difficult to manage patients (Kirby, 2006), and/or the use of the criminal justice system to manage difficult patients (Jansman-Hart et al., 2011). That being said, there is evidence to suggest that it is easier for someone with a severe mental illness (SMI) to access mental health care after they have been charged with a criminal offence as opposed to accessing these services prior to such an event through the civil psychiatric system (Crocker et al., 2015).  This trend toward forensification of mental health services risks criminalizing mental illness.  This clearly has important implications for the civil rights of these individuals and also carries with it economic and social implications. For instance, forensic inpatient care is nearly five times more costly than civil psychiatric inpatient care (Institute of Health Economics (IHE), 2014).  In addition, patients in the forensic system face the double stigma of criminal justice 6  involvement as well as having a mental illness, making community reintegration that much more challenging, specifically finding housing and employment (Salem, Crocker, Charette, Seto, Nicholls, & Côté, 2014).   In Canada, people enter the forensic system after being charged with a criminal offence and being found unfit to stand trial or after being found not criminally responsible on account of a mental disorder (NCRMD).  Involuntary patients enter the civil psychiatric system after being certified by two doctors that they are suffering from a mental illness and are a present danger to themselves or others.  It is quite possible, for example, for a mentally ill individual who is behaving in an undesirable, but non-violent manner, to be arrested and changed with a crime and be processed by the forensic system, or, in contrast, to be taken to an emergency room, involuntarily committed and cared for in the civil psychiatric system, or alternatively released into the community (Charette, Crocker, & Billette, 2014).  The overlap between these systems, and the excessive transferring of care from one system to another leads to disrupted and fractured patient care.  Thus, better understanding these two populations is important from a public safety, societal, economic, human justice, and practice perspective. There is a dearth of literature comparing forensic and civil psychiatric samples. Amongst the few studies conducted, Heilbrun et al. (1995) focused primarily on seclusion and restraint; Seto et al. (2004) examined criminogenic, clinical, and social problems of the two groups; Dumont et al. (2012) studied clinical characteristics of both groups that acted as obstacles in the transition to community-based care; and Landgraf et al. (2013) compared clinical and demographic characteristics of female forensic and civil psychiatric patients.  Results from these studies had very little overlap.  Understanding the psychosocial needs and risk profile of civil 7  psychiatric and forensic psychiatric patients and the prevalence of negative outcomes of concern in these two populations is essential for informing policy and practice.  1.3 Patient Safety, Risk Assessment, and Risk Management Historically, risk assessment focused on the unstructured clinical judgement of “dangerousness,” or the whether or not a person should be held in a secure facility because they are a danger to society (Jackson& Guyton, 2008).  More recently, risk assessment has focused on the prediction of long-term risk for violence based on statistically derived risk factors (Cooper, Griesel, & Yuille, 2007).  Subsequently, the risk assessment field expanded to include structured professional risk judgement (SPJ) approaches which incorporate statistically informed risk factors and clinical judgement.  Best Practice in Managing Risk (Department of Health, 2007) details 16 best practice guidelines for effective clinical risk assessment and risk management within mental health services.  The guidelines endorse the structured professional judgement (SPJ) approach to risk assessment and risk management.  Until recently there has been a clear and overwhelming tendency to focus on risks, deficits, or vulnerabilities in the risk assessment field (Rogers, 2000) with few SPJ guides available to help clinicians construct positive treatment plans and to ensure their assessments address these ethical and professional recommendations.  START is one of the few measures that satisfies the requirement to address clients’ strengths as well as their deficits (also see Structured Assessment of Violence Risk in Youth (SAVRY) Borum, Bartel, & Forth, 2006; Inventory of Offender Risk, Needs, and Strengths (IORNS), Miller, 2006; Structured Assessment of Protective Factors for Violence Risk (SAPROF) de Vogel, de Ruiter, Bouman, & de Vries Robbé, 2009).  START is also unique relative to existing violence risk assessment instruments because it guides clinicians to consider historical markers, as well as both vulnerabilities and strengths, simultaneously, on 20 dynamic items.  START is 8  intended to inform clinical interventions and assist in treatment and risk management efforts aimed at averting the types of adverse events that occur at elevated rates in populations of mentally and personality disordered individuals (violence, self-harm, suicide, substance abuse, unauthorized leave, self-neglect, and being victimized).  The measure was developed with the intention of application to evaluations and care planning with persons living with mental illness and personality disorders in diverse settings (inpatient and community), and across populations (corrections, forensic and civil psychiatry).  Despite the novel aspects and strengths of START outlined above that make it a particularly suitable tool for risk assessment and risk management in this population, research on the measure is still accumulating.  For example, a recent meta-analysis of the START literature summarized several deficits in the current START research including:  low predictive accuracy for self-neglect and victimization, lack of information on the variability of scores over time, and its use as an intervention to reduce violence in a forensic outpatients demonstrated no change in subsequent violence rates (O’Shea & Dickens, 2014).  START has been incorporated into clinical practice with considerable enthusiasm.  It has been implemented in more than 15 countries and translated into eight languages.  Despite being a relatively new measure, the literature on the START is beginning to grow.  Doyle, Lewis, and Brisbane (2008) evaluated the practical utility and face validity of the START amongst forensic mental health nursing staff in Manchester, UK.  The majority of respondents reported that the measure assisted in their individual risk formulations, with 85% endorsing ratings from moderate to very useful.  Desmarais, Nicholls, and Brink (2007) gauged user-satisfaction on the START via surveys administered to clinical staff at the British Columbia Forensic Psychiatric Hospital (FPH).  Regardless of rater profession, clinicians endorsed the user friendliness items at levels of 80% or higher, with the exception of only one item (finer scoring distinctions: 56%).  A high 9  level of endorsement was reported concerning the measure’s clinical utility; specifically, 93% of respondents agreed START is a clinically useful tool.  Kroppan, Nesset, Pedersen, Almvik, and Palmstierna (2011) examined the implementation of START in a forensic high security unit in Norway.  They found that 73% of staff agreed that START contributed to enhancing a more systematic approach to risk assessment and risk management.  An additional 79% of direct-care providers agreed that the START was useful as a tool in risk assessments and treatment planning.  From a study conducted in Quebec, Canada; Crocker, Braithwaite, Laferriere, Gagnon, Venegas, and Jenkins (2011) reported the results of a longitudinal prospective mixed method (qualitative and quantitative) implementation study.  Their results indicated START was well-integrated into the unit’s clinical and administrative activities (Crocker et al., 2011).  Research also indicates that START has strong psychometric properties.  Nicholls and colleagues (2006) evaluated START assessments completed by nurses, social workers, and psychiatrists in a sample of 137 male forensic psychiatric patients.  The study indicated excellent interrater agreement overall, intraclass correlation coefficient (ICC) = .87, as well as when it was examined by the profession of the evaluator (ICCs for nursing = .88, social work = .92, and psychiatry = .80, all p < .001).  Receiver Operating Characteristic (ROC) analyses of a subsample of 50 patients who remained hospitalized throughout follow-up revealed good validity in predicting both verbal and physical aggression, all in the inpatient setting (Area Under the Curve (AUC) > .67, p < .05). In another Canadian study, Brathwaite, Charette, Crocker, and Reyes (2010) completed a longitudinal prospective mixed-method study to evaluate whether the START was associated with adverse events that occurred over the short-term (30 days) and reported that the vulnerability scale significantly predicted physical aggression against others (AUC = .66, p < .05).  A recent analysis of assessments completed on 30 male forensic 10  psychiatric inpatients provided evidence for the ability of the START Version 1.11 to predict short-term violence risk (Wilson et al., 2010).  Specifically, results demonstrated that Strength and Vulnerability total scores, as well as the final risk estimates, predicted aggressive behaviour in the three months following the assessment (AUC > .73, all p < .001).  Finally, Desmarais, Nicholls, Wilson, and Brink (2011) examined the predictive and incremental validity of research assistant completed START assessments for 120 male forensic psychiatric patients.  START evidenced excellent interrater reliability and demonstrated both predictive and incremental validity over other established risk assessment measures.   Research on the START, and for that matter most structured professional judgment measures, has focused to a large extent on assessments completed by research assistants (e.g., Desmarais et al., 2011; Wilson et al., 2011).  Although the START has now been implemented across several large mental health systems (e.g., Oregon State Hospital, USA; St. Andrews Healthcare, Northampton, UK, and the British Columbia Forensic Psychiatric Service in Canada), little effort has been made to evaluate the use of the scheme in clinical practice.  To date, research has focused to a large degree on the predictive accuracy of START and has neglected the success of clinical implementations (or field reliability).   There have been only a handful of studies which have considered the psychometric properties of START in a civil psychiatric population or considered outcomes other than violence.  Additionally, to date, the existing research has largely failed to consider civil samples or outcomes other than violence, has been limited to small sample sizes and/or has not considered the utility of risk estimates, the most fundamental part of a structured professional judgment (SPJ) measure.  For example, Gray et al. (2011) (N44) included both civil and forensic inpatients in their research and found that START summary risk judgments are a good predictor 11  of all outcome behaviours they assessed (violence to others, self-harm, self-neglect, and being victimized).  By comparison, strength and vulnerability total scores were only predictive of violence, providing support for the value of structured professional judgment model of violence risk assessment (Gray et al., 2011).  Abidin et al. (2013) conducted research in a forensic setting and found START strength and vulnerability scores to be predictive of violence, however, neither were predictive of self-harm.  Unfortunately, Abidin et al. did not use the START as intended, by examining the association between the summary risk estimates and the outcomes.  A recent Canadian study considered the psychometric properties of START in a civil psychiatric hospital and found START strength, vulnerability and risk estimates to be variably predictive of aggression toward others, self-harm, suicidality, unauthorized leave, substance abuse, self-neglect and victimization (Braithwaite et al., 2010; (n = 34).  Furthermore, a current systematic review and meta-analysis of START identified the need for additional research encompassing the complete range of outcomes predicted by START and noted the limitations in the diversity of populations used in the validation research (O’Shea & Dickens, 2014).  1.4 Summary Chapter two of this dissertation contains a paper that was published in Schizophrenia Research (Petersen et al., 2013).  This study was designed as a large-scale, prospective program evaluation of the RVH Redevelopment Project.  The results report on a cohort of patients who moved from RVH to community-based facilities in the Vancouver Island, Interior, Northern, and Fraser Health Authorities between 2001 and 2004.  The objectives of this chapter are to provide a description of the study sample and methodology, report the variety and characteristics of the 12  facilities participants moved to over the course of the RVH closure, and describe changes in clinical and psychosocial characteristics of participants over a 5-year period.  Chapter three consists of a paper submitted to Psychiatric Services, that was designed to consider the potential overlap between forensic and civil psychiatric patients by comparing and contrasting the psychosocial, clinical, and risk profile of these two groups.  In addition, the prevalence of a variety of negative outcomes (i.e., aggression, inappropriate sexual behaviour, self-harm, suicide, self-neglect, unauthorized leave, substance abuse, victimization, and stalking) in a six month time frame are compared.   Chapter four consists of a validation of START in forensic mental health practice published in the International Journal of Forensic Mental Health (Nicholls et. al., 2011).  It addresses three specific research questions:  (1) What is the clinical and risk profile of a large Canadian forensic psychiatric cohort?  (2) What are the psychometric properties of the START in a forensic service when the evaluations are completed by clinical teams?  (3) How are these profiles and START outcomes similar or different amongst male and female patients and across various settings (i.e., hospital versus clinic) or security levels (i.e., maximum, medium, minimum)?   Chapter five consists of a paper submitted to Assessment and compares the psychometric properties of START in forensic and civil psychiatric sample.  Previous research has demonstrated that it is hard to statistically model START strength and vulnerability scores together (e.g., Viljoen et al., 2011; Wilson et al., 2010;) due to the high correlations between the two dimensions that lead to multicolinearity.  An exploration of an alternative approach, cluster analysis, was therefore conducted.  A cluster analytic approach could prove helpful by providing clinicians with information on how strength and vulnerability scores function together 13  holistically, which may aid in their decision-making.  From a research prospective, producing a unified model including both strength and vulnerability scores that could be tested would further our understanding of the validity of START.    14  Chapter 2:   Redevelopment of Tertiary Psychiatric Services in British Columbia:  A Prospective Study of Clinical, Social, and Residential Outcomes of Former Long-Stay Inpatients1  2.1 Redevelopment of Psychiatric Services in British Columbia:  Prospective Study of Clinical, Social and Residential Outcomes Deinstitutionalization of severely and persistently mentally ill individuals has been one of the most significant developments in mental health policy and practice in the last 50 years.  Recently, deinstitutionalization has received considerable attention in the media and is often prominent on political agendas (Bryeton, 2006; Grindlay, 2009; Smyth, 2006).  This has been particularly true in British Columbia, where the only tertiary psychiatric hospital has been downsizing for several decades (Morrow et al., 2008).  Many researchers have asserted that community-based care is inadequate and leads to negative outcomes such as homelessness, inadequate treatment, and/or transinstitutionalization (i.e., patient care transferred to correctional facilities, or forensic psychiatric facilities) or death (Bachrach, 2001; Lamb & Bachrach, 2001).  However, these unfavourable results have been attributed to underfunding and poor management at the system level rather than limitations with community-based care models (World Health Organization, 2003). Despite the academic debates and popular discourse focusing on perceived obstacles and evidence-based challenges to deinstitutionalization, there is evidence that replacing long-stay                                                  1 Petersen, K. L., Nicholls, T. L., Groden, D., Schmitz, N., Stip, E., Goldner, E. M., & ... Lesage, A. (2013). Redevelopment of tertiary psychiatric services in British Columbia: A prospective study of clinical, social, and residential outcomes of former long-stay inpatients. Schizophrenia Research, 149(1-3), 96-103. doi:10.1016/j.schres.2013.05.022 15  inpatient beds in large scale psychiatric hospitals with community-based residential facilities leads to positive outcomes when properly managed.  Several programs of research examining hospital closures and transitions to community-based care have been conducted in diverse countries have been evaluated, for example:  Italy (Barbato, 1998; Lesage & Tansella, 1993; D’Avanzo et al., 2003), the United States (e.g., Indiana) (McGrew et al., 1999a, 1999b; Pescosolido et al., 1999), the United Kingdom (UK) (Leff et al., 2000; Leff & Trieman, 2000), Australia (Hobbs et al., 2000; Newton et al., 2000; Hobbs et al., 2002), Canada (e.g., Quebec) (Trudel & Lesage, 2006; Lesage et al., 2000), and the Netherlands (Amsterdam) (Duurkoop & van Dyck, 2003).  Each of these studies has demonstrated that properly executed programs of deinstitutionalization are feasible and do not lead to negative clinical or social outcomes.  Specifically, perhaps the most important longitudinal study in this field of research is the Team for the Assessment of Psychiatric Services (TAPS) project conducted in the UK (Leff et al., 2000).  With results that generally echo other studies, the TAPS study found no amelioration of participants’ clinical profiles or undesirable social behaviours; however, improvements in domestic living skills, community living skills, and quality of life were reported.  In addition, patients developed more friendships and enjoyed increased freedom after leaving hospital and relocating to the community.  The TAPS study showed no increase in mortality, homelessness, or criminal activity among its participants (Leff et al., 2000; Leff & Trieman, 2000; Leff et al., 1996; Dayson, 1993; Leff et al., 1994).  Over and above clinical and social-cultural evidence that community-based care is the best practice, there is also evidence that it is a cost-effect method of service delivery (Reinharz et al., 2000).  It is these positive results that have encouraged provinces, such as British Columbia (BC), to pursue the redevelopment of psychiatric care and the closure of its only tertiary psychiatric hospital.  Despite positive results from earlier studies, 16  one significant question remains, will the redevelopment of psychiatric services with a more handicapped population be achieved with similar effectiveness (i.e. clinical and social outcomes) and efficiency (i.e. at similar or lower costs) in the Canadian context. Patients who remain in long-stay inpatient hospitals have more severe and persistent mental disorders and greater levels of disability than the patients who were deinstitutionalized before them (Knapp et al, 2011).  It has been argued that as patients are moved into the community, less disabled patients are ‘creamed off’ and the most difficult to place patients form a ‘remnant group’ (Ford, 1987).  This trend was supported by findings from the TAPS project which found a selection bias such that patients who initially left the hospitals were younger, had spent less time in psychiatric hospitals, had fewer social problems, had larger social networks, and were less likely to be diagnosed with schizophrenia (Jones, 1993).  As deinstitutionalization has progressed in many countries, several authors have estimated that about 10% of severely mentally ill patients (the ‘remnant group’) require more comprehensive care than can typically be provided in the community (Gudeman & Shore, 1984; Trauer et al., 2001).  Numerous countries have developed parallel strategies to address the needs of this unique sub-population.  For example, the UK has developed ‘hostel wards’ and Australia has established ‘community care units,’ both of which are intended to be home-like facilities providing intensive treatment and rehabilitation in normalized living conditions as an alternative to long-term hospitalisation (Garety & Morris, 1984; Wykes & Wing, 1982). In British Columbia (BC), a province of approximately 4.6 million people, as in many industrialized countries, psychiatric services have been undergoing reorganization and redevelopment since the 1960s (BC Government, 2012).  Although this process was far from linear and was punctuated by various delays and interruptions, Riverview Hospital (RVH), BC’s 17  only tertiary care psychiatric hospital, underwent downsizing from approximately 4,000 beds in the early 1960s to approximately 500 beds in the early 1990s (Macfarlane et al., 1997; BC Mental Health and Addiction Services, 2010).  The current phase of the RVH Redevelopment Project, which began in 2002, was instigated by the release of a new Mental Health Plan for BC in 1998.  The BC Mental Health Plan called for the continued redevelopment and decentralization of Riverview Hospital and the building or renovating of small purpose-designed, community-based facilities.  In addition, in 2002, BC health services were reorganized into five geographically-based regions, each responsible for the health care needs of their populations (Priebe & Turner, 2003; BC Ministry of Heath, 1998).  The Ministry of Health committed CAN$138 million in capital funding to construct new facilities or renovate existing facilities where appropriate.  Additionally, CAN$4,050,000 of annualized funding was pledged (BC Mental Health and Addiction Services, 2009).  The current and final phase of the RVH Redevelopment Project is anticipated to be complete in 2012.  In a process that parallels that of other countries, BC is developing regionalized Tertiary Psychiatric Residential Facilities (TPRFs).  These facilities are intended to be smaller and more home-like than hospitals and provide intensive long-term treatment and rehabilitation intended to support recovery for tertiary psychiatric patients.  The funding and construction of the facilities necessary to receive the final ~500 RVH patients has taken place over more than eight years.   The state of the field demonstrates that there is still much to be learned about deinstitutionalization and community-based care, warranting further research (Priebe & Turner, 2003).  This study was designed as a large-scale, prospective program evaluation of the RVH Redevelopment Project.  The results report on a cohort of patients who moved out of RVH and into TPRFs in the Vancouver Island, Interior, Northern, and Fraser Health Authorities between 18  2001 and 2004.  The objectives of this first paper are to provide a description of the study population and methodology, to report the variety and characteristics of the facilities participants moved to over the course of the study, and to describe the baseline clinical and psychosocial characteristics and associated outcomes over a 5-year period.   2.2 Method The research was approved by the research ethics committees of the University of British Columbia, Simon Fraser University, Riverview Hospital and all participating regional health authorities.  2.2.1 Design This study used a prospective cohort design, comparing measurements across multiple time points. The goal was to examine the outcomes of a cohort of 189 long-stay patients at Riverview Hospital (RVH) in British Columbia, Canada, some of whom moved into TPRFs, some relocated into the community in less structured facilities, and some remained at RVH.  Most patients who remained at RVH did so generally because their region of origin was not yet scheduled for redevelopment.    2.2.2 Procedures Data was collected from clinical files at RVH and each participating site, semi-structured interviews and self-report measures were completed with patients and interviews with also conducted with staff members.  Baseline data collection was conducted at RVH and data collection was repeated annually for a period of five years in the location of the participant’s 19  residence at the time.  Residential facility results report the location of the participant at each follow-up.  If a participant was unavailable for a follow-up interview due to, for example, physical illness, the location of the participant was recorded and the characteristics of the facility ascertained.    Approximately 17% of our participants refused to participate at some point in the study.  Statistical analysis indicated not difference in baseline characteristics of participants who dropped-out.   2.2.3 Measures Canadian Toolkit:  This instrument includes measures of sociodemographic characteristics (e.g.: age, gender, marital status, and work history) as well as residential, financial, legal, and diagnostic information (Ontario Federation of Community Mental Health,1999; Piat et al., 2008; Sheldon et al., 2006). Brief Psychiatric Rating Scale, Version 4 (BPRS):  The BPRS is a widely used scale that provides a means of assessing current mental-health status.  Items are rated from 1 to 7 based on the participants’ self-report as well as observation of the participant’s behaviour and speech.  The scale has well-documented reliability and validity across similar samples, and is responsive to change (Velligan et al., 2005; Ventura et al., 1993; Thomas et al, 2004). Social Behaviour Scale (SBS):  The SBS assesses 21 abnormal behaviours and psychiatric symptoms ranging from hygiene problems to socially unacceptable behaviour.  Information is obtained from staff interviews.  The SBS has well-documented reliability and validity across multiple similar long-stay patient samples (Wykes & Sturt, 1986; Salvador-Carulla et al., 1998).   Independent Living Skills Schedule (ILSS):  This schedule includes 70 items covering several domains (e.g. appearance and clothing, personal hygiene, care of personal possessions, and food 20  preparation).  Information is collected from subject interviews and corroborated by staff interviews.  The ILSS has been show to have acceptable psychometric properties (Wallace et al., 2000; Perivoliotis et al., 2004); Cyr et al., 1994).   Cambridge Neuropsychological Test Automated Battery (CANTAB):  CANTAB is a set of neuropsychological tests administered using a computer with a touch-sensitive screen.  The subtests included in our protocol were the Motor Screening (MOT), Spatial Recognition Memory and Paired Associates Learning tests.  The reliability, validity, and sensitivity to changes of CANTAB had been supported by various studies in multiple populations (Prouteau et al, 2004; Levaux et al, 2007; Potvin et al, 2005). Level of Care Survey (LOCS):  The LOCS has been utilized by multiple agencies in both Canada and the United States to assess level of functioning, clinical needs, and discharge readiness of patients experiencing mental illness (Fabisiak et al., 1983; Lambert, 1982; Aviram et al., 1995).  The LOCS was utilized in this study to assess level of physical health/disability.  Wisconsin Quality of Life Index (W-QLI):  The W-QLI is a multidimensional tool developed to assess quality of life among individuals experiencing mental illness.  The W-QLI was employed to assess overall life satisfaction and satisfaction with social relations.  The W-QLI has been shown to have acceptable psychometric properties (Diamond & Becker, 1999; Becker, 1998; Malla et al., 2006). Riverview Patient Inventory:  The RPI is a brief 36-item scale designed to assess problem behaviours that have an effect on treatment and community placement.  It was developed to be a quick and convenient tool for nurses and other care-givers as a means of obtaining a comprehensive assessment.  This tool has been utilized in a limited number of studies; however, it has promising psychometric properties (Haley et al., 2002; Trudel & Lesage, 2006). 21   2.2.4 Settings All participants underwent baseline evaluations at Riverview Hospital (RVH).  RVH is British Columbia’s only large scale psychiatric hospital.  It provides multifaceted care for patients housed in a traditional institutional setting within a large park-like campus.  To achieve the first objective of describing the settings where patients were relocated over the course of RVH redevelopment, all facilities were coded using three classification systems.  The first classification system was closely related to BC’s administrative description of facilities being differentiated on various physical and program of care factors (see Table 1).  The second classification system used was the European Services Mapping Schedule (ESMS).  The ESMS was introduced in 2000 and has been used frequently to compare mental health services in locations around the world (Johnson & Kuhlmann, 2000; Salvador-Carulla et al., 2000).  The third classification system was based on structural facility descriptors as suggested by Shepherd et al. (1995) and shown in Table 4.  Table 2.1  Facility Characteristics Facility Name Facility Characteristics Forensic Facilities Treatment and assessment for mentally ill adults who have come into conflict with the law. Correctional Facilities  Incarceration of individuals on remand, awaiting trial, or receiving a custodial sentence. Acute or Intensive Care Units Inpatient psychiatric care.  These wards often are housed within a general hospital. Extended Care / Intermediate Care Facilities Long-term care for patients who primarily require nursing care.  These include nursing and elder care facilities. Tertiary Psychiatric Residential Facilities (TPRFs) Purpose built or renovated facilities for patients with severe and persistent mental disorders that cannot be cared for in lower intensity residential facilities or supporting housing alternatives.  They provide 24 hour multidisciplinary staffing with staffing ratios comparable to acute care wards.  Programming is focused on both clinical and rehabilitation care.  In addition, these facilities are relatively small (usually less than 25 beds) and are run by the local health authority 22  Facility Name Facility Characteristics Community Psychiatric Residential Facilities (CPRFs) Purpose built or renovated residential homes providing accommodation for individuals with severe and persistent mental disorders.  Programming is focused on both clinical care and rehabilitation.  These facilities may be operated by the local health authority or by community organizations on behalf of the local health authority Mixed Facilities Provide care for patients primarily requiring nursing care and /or psychiatric care and rehabilitation.  These two groups of patients are most often housed on different floors or on specialized wards. Supervised Private Family Boarding Homes Converted homes in residential areas, housing individuals with mental illness.  These homes are operated by a private owner or community-based organization and supervised by the local health authority or social service agency.  Meals are provided and programming is focused on activities of daily living rather than clinical care and rehabilitation. Semi-Independent Apartment Living Self-contained suites with mental health staff available on-site to provide medication monitoring, facilitate access to community resources, etc. as required. Independent Living Living alone, with family or with a spouse or partner and without onsite services. Single Room Occupancy Hotels (SROs) Hotel rooms which often are substandard and in marginalized areas of the community. Homeless Shelters Provide emergency or short-term shelter for homeless individuals.   2.2.5 Analysis  Data analysis was conducted using SPSS (version 16.0).  The statistics computed were descriptive in nature (percentage, mean, standard deviation).  Comparisons were computed with parametric tests for paired samples.  Clinical outcomes are based on the sub-sample of participants for whom both baseline and five-year follow-up data was available.  2.3 Results 2.3.1 Sample All original participants were followed for the 5-year period, although not all accepted or could be interviewed, and 10.6% died in the course of the study (see Table 4). The majority of the participants were male (60.3%), Caucasian (86.2%) and their mean age was 47 years (SD = 23  11.51; range =22-80).  Most participants had not completed high school (71.5%), were in the hospital on an involuntary basis (81.5%), had been hospitalized more than six times (67.2%), and the index hospitalization generally exceeded two years (66.7%).  The predominant diagnosis was a schizophrenia spectrum disorder (79.9%) and a significant number of participants had a dual diagnosis (52.4% substance abuse, 22.2% developmental disability) (see Table 2).  Table 2.2  Sociodemographic and Clinic Characteristics at Baseline Demographic Characteristics Sample (N=189) Number Percentage Gender      Men      Women  114 75  60.3 39.7 Age      Mean (SD)      Range  47.28 (11.51) 22 – 80  Ethnicity      Caucasian      Aboriginal      Asian (East, South, West)      Other  163 19 6 1  86.2 10.1 3.2 0.5 Relationship Status      Single      Married      Separated/Divorce/Widowed  146 4 39  77.2 2.1 20.6 Education      Elementary Only (≤ 7th grade)      Some High School (7-11th grade)      High School Graduate / GED      Some Post Secondary  9 126 35 19  4.8 66.7 18.5 10.1 Age at First Psychiatric Hospitalization      10 – 20      21 – 30      31 – 40     > 40  89 70 20 9  47.3 37.2 10.6 4.8 Number of Previous Hospitalizations      1 – 5      6 – 10     11 – 20      > 21      Unknown  62 50 30 12 35  32.8 26.5 15.9 6.3 18.5 Length of Index Hospitalization (months)      ≤ 12      13 – 24      25 – 36      ≥ 37  31 32 30 96  16.4 16.9 15.9 50.8 Diagnosis (Axis 1)      Schizophrenia Spectrum      Other  151 38  79.9 20.1 24  Demographic Characteristics Sample (N=189) Number Percentage Dual Diagnosis       Substance Abuse      Developmental Disability  99 42  52.4 22.2 Employment       No employment at time of admission  147  77.8 Legal Status      Involuntary      Voluntary      Unknown  154 19 16  81.5 10.1 8.5 File-based Evidence of Criminal History      Any evidence criminal history      Any arrests or criminal convictions      Any violent behaviour      Any arrests or criminal convictions for violent offences  144 101 105 67  76.2 53.4 55.6 35.4   2.3.2 Clinical and Psycho-Social Results Results from the BPRS (t(88)=-1.29, p=0.202, d=0.17) and the psychological symptoms subscale of the RPI (t(100)=-2.01, p=0.047, d=0.25) indicate that study participants did not experience any significant changes in psychiatric symptoms captured by these measures five years after their baseline assessment (see Table 3).   Several measures were used to assess changes in participants’ problem behaviours.  Results from the SBS, a global measure of problems with behaviours in multiple areas from attention seeking behaviour to sexually inappropriate behaviour, indicate that participants experienced an increase in both mild and severe problem behaviours from baseline to follow-up at five years (severe behaviour problem score:  t(103)=-3.38, p=0.001, d=0.40; mild and severe problem score:  t(103)=-4.82, p=0.000, d=0.56).  In contrast, RPI results indicate no change in aggressive behaviours (RPI aggressive behaviours subscale:  t(102)=-0.08, p=0.993, d=0.00) (see Table 3).  CANTAB neuropsychological results demonstrate improvements in some areas and no significant change from baseline to the five year follow-up in other areas.  SRM number correct 25  (t(59)=2.22, p=0.030, d=0.33) and PAL total errors (t(58)=3.59, p=0.001, d=0.63) decreased significantly.  MOT mean latency (t(62)=-1.52, p=0.132, d=0.26) and PAL total errors (adjusted) (t(58)=-1.91, p=0.061, d=0.22) did not demonstrate significant changes (see Table 3).  While positive psychiatric symptoms remained static, participants experienced a significant increase in quality of life (W-QLI total score:  t(54)=-2.51. p=0.015, d=0.38) and their overall life satisfaction (W-QLI overall satisfaction sub-scale:  t(61)=-2.66, p=0.010, d=0.41).  However, satisfaction with social relations (t(52)=1.19, p=0.241, d=0.20) did not change significantly (see Table 3). There were significant gains in independent living skills (ILSS total score:  t(104)=-15.72, p=0.000, d=2.27), which could also be translated into less hypoactivity, one characteristic of the negative syndrome of schizophrenia.  Specifically, statistically significant gains were observed in care of personal possessions (T(103)=-13.69, p<0.001, d=2.12), food preparation and storage (t(103)=-24.44, p<0.001, d=3.76), money management (t(103)=-28.59, p<0.001, d=3.96), transportation (t(103)=-11.19 p<0.001, d=1.66), leisure and recreation (t(103)=-4.00, p<0.001, d=0.50); job-seeking skills (t(76)=2.70, p=0.009, d=0.54), and job maintenance skills (t(55)=-3.351, p=0.001, d=0.51).  There were no observed differences in several areas including personal appearance (t(103)=-1.33, p=0.186), d=0.15), and hygiene (t(103)=-0.58, p=0.563, d=0.07).  A significant decrease was found in health maintenance (t(103)=5.47, p=0.000, d=0.58).  Interestingly, the number of physical health problems experienced by participants decreased significantly (t(103)=3.44, p=0.001, d=0.38) (see Table 3). 26  Table 2.3  Clinical and Psychosocial Outcomes Measures (μ, sd, n) Baseline  (Complete Sample) Baseline (T-Test Sample) Follow-up #5 (Complete Sample) Follow-up #5 (T-Test Sample) t (p-value)  Brief Psychiatric Rating Scale      Total score  (N=162) 53.19 (10.32) (n=89) 52.68 (10.39) (N=102) 54.74 (13.14) (n=89) 54.75 (13.63)  1.29 (0.20) Social Behaviour Schedule       Total score            Severe behaviour problems score            Mild and severe behaviour problems            score (N=189) 14.98 (7.81) 1.71 (1.77) 4.25 (2.93) (n=104) 14.09 (7.69) 1.65 (1.82) 3.89 (2.81) (N=104) 17.93 (8.76) 2.40 (1.92) 5.71 (3.63) (n=104) 17.93 (8.76) 2.40 (1.92) 5.71 (3.63)  4.36 (0.00) 3.38 (0.01) 4.82 (0.00) Independent Living Skills Schedule      Total score           Personal appearance           Personal hygiene           Care of personal possessions           Food preparation and storage            Health maintenance           Money management           Transportation           Leisure and recreation           Job-seeking skills           Job-maintenance skills (N=189) 74.25 (6.15) 10.65 (2.04 13.58 (2.18) 4.78 (1.20) 1.11 (0.89) 8.39 (1.25)  3.38 (0.72) 6.35 (1.03) 18.19 (2.17) 7.14 (2.47) 0.69 (1.31)  74.18 (5.84) [105] 10.43 (1.87) [104] 13.60 (2.11) [104] 4.72 (1.53) [104] 1.19 (1.19) [104] 8.38 (1.27) [104] 3.40 (0.74) [104] 6.42 (0.93) [104] 17.86 (2.15) [104] 7.99 (0.11) [77] 1.07 (1.45) [56]  97.16 (14.40)[105] 10.68 (1.57) [104] 13.74 (1.75) [104] 9.25 (3.12) [104]  9.61 (3.28) [104] 7.63 (1.26) [104] 8.30 (1.73) [104] 8.54 (1.62) [104] 18.91 (2.07) [104] 7.77(0.71) [77] 1.80(1.44) [56]  97.16 (14.40)[105] 10.68 (1.57) [104] 13.74 (1.75) [104] 9.25 (3.12) [104]  9.61 (3.28) [104] 7.63 (1.26) [104] 8.30 (1.73) [104] 8.54 (1.62) [104] 18.91 (2.07) [104] 7.77(0.71) [77] 1.80(1.44) [56]  15.72 (0.00) 1.33 (0.19) 0.06 (0.56) 13.39 (0.00) 24.42 (0.00) 5.47 (0.00) 28.59 (0.00) 11.19 (0.00) 4.00 (0.00) 2.70 (0.01) 3.35 (0.00) CANTAB      MOT mean latency      SRM number correct      PAL total errors      PAL total trials (adjusted)  1271.62 (7.55) [145] 11.10 (3.92) [129] 14.72 (14.56) [140] 34.15 (19.37) [140]  1181.59 (741.19) [63] 11.22 (3.95) [60] 16.71 (17.60) [59] 33.08 (19.72) [59]  1332.00 (603.20) [74] 9.29 (6.35) [76] 8.94 (6.77) [71] 35.80 (16.27) [71]  1356.28 (615.19) [63] 9.52 (6.21) [60] 8.86 (7.22) [59] 37.22 (17.21) [59]  1.52 (0.13) 2.20 (0.03) 3.59 (0.00) 1.91 (0.06) Physical Disorders and Disabilities Levels of Care Survey      # of physical health problems (N=189)  2.44 (2.12) (n=104)  2.13 (2.02) (N=104)  1.46 (1.49) (n=104)  1.46 (1.49)   3.41 (0.00) 27  Measures (μ, sd, n) Baseline  (Complete Sample) Baseline (T-Test Sample) Follow-up #5 (Complete Sample) Follow-up #5 (T-Test Sample) t (p-value)  Wisconsin Quality of Life Index for Mental Health      Total score*         Overall satisfaction*         Satisfaction with social relations*  9.09 (17.10) [116] 6.09 (11.96) [123] 2.86 (7.44) [116]  9.19 (18.19) [55] 7.48 (12.47) [62] 2.67 (7.22) [53]  16.56 (19.54) [77] 12.72 (12.92) [81] 3.46 (7.82) [76]  16.20 (18.40) [55] 12.54 (12.35) [62] 4.02 (6.64) [53]  2.54 (0.02) 2.66 (0.01) 1.19 (0.24) Riverview Patient Inventory      Total score         Daily routine         Psychological symptoms         Social interaction         Aggressive behaviours         Community preparation (N=189) 101.84 (25.31) 27.60 (8.47) 24.00 (6.60) 14.06 (4.59) 12.22 (4.38) 24.01 (7.28)  99.54 (26.05) [101] 27.05 (8.73) [101] 23.38 (6.64) [101] 13.56 (4.54) [103] 12.13 (4.73) [103] 23.86 (7.79) [78]  106.25 (29.46) [101] 29.99 (9.16) [101] 25.24 (8.49) [101] 15.22 (5.09) [103] 12.14 (4.91) [103] 24.08 (8.30) [79]  106.25 (29.46) [101] 29.99 (9.16) [101] 25.24 (8.49) [101] 15.22 (5.09) [103] 12.14 (4.91) [103] 23.95 (8.28) [78]  2.16 (0.03) 2.97 (0.01) 2.01 (0.05) 2.78 (0.01) 0.03 (0.98) 0.08 (0.93)  * Standardized Scores 28  2.3.3 Facility Characteristics There was no evidence of judiciary transinstitutionalization or homelessness.  Throughout the follow-up period there was only one participant residing at a correctional facility, one participant residing in a forensic facility, one participant who had spent some time homeless, and one participant who lived in a single room occupancy hotel (SRO) (see Table 1; Figure 1).  The demonstrated residential trend is a slow movement of participants towards facilities with a lesser degree of institutionalization.  For example, 13% of participants were living in facilities with 25 or fewer residents at the time of the first follow-up and 32% were living in these facilities at follow-up five.  Eighty nine percent of participants lived in facilities with on-site programming at follow-up one compared to only 67% at follow-up five.  Despite this trend, the majority (70.7%) of participants remained in facilities with 24 hour staffing at follow-up five (see Table 4).  The percentage of participants residing at intensive care facilities and TPRFs decreased over time (e.g., TPRF:  follow-up #1 – 47.1% and follow-up #5 – 27.5%).  In contrast, participants residing in CPRFs, semi-independent apartment living, independent living, and extended/intermediate care facilities increased over time (e.g., extended/intermediate care:  follow-up #1 – 1.16% and follow-up #5 – 10.6%) (see Figure 1). 29  Table 2.4  Assessment Facilities N(%) FU #1 (n=189) FU #2 (n=189) FU #3 (n=189) FU #4 (n=189) FU #5 (n=189) Classification – European Services Mapping Schedule      Residential           Secure           Acute                Hospital                Non-hospital           Non-acute                Hospital                     Time Limited                           24 h support                            Daily support                     Indefinite stay                           24 h support                            Daily support                            Lower support                Non-hospital                     Time Limited                           24 h support                            Daily support                     Indefinite stay                           24 h support                            Daily support                            Lower support      Outpatient           Continuing Care                Mobile                     Low                     Moderate                     High                Non-Mobile                     Low                     Moderate                     High      Long-term care      General hospital      Deceased      Temporary non-mental health specific housing      Unknown*  174 (92.1) 0 (0.0) 69 (36.5) 69 (36.5) 0 (0.0) 105 (92.1) 26 (13.8) 0 (0.0) 0 (0.0) 0 (0.0) 26 (13.8) 26 (13.8) 0 (0.0) 0 (0.0) 79 (41.8) 0 (0.0) 0 (0.0) 0 (0.0) 79 (41.8) 70 (37.0) 9 (4.8) 0 (0.0) 6 (3.2) 6 (3.2) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 6 (3.2) 4 (2.1) 2 (1.1) 0 (0.0) 2 (1.1) 3 (1.6) 1 (0.5) 0 (0.0) 3 (1.6)  164 (86.8) 1 (0.5) 51 (27.0) 51 (27.0) 0 (0.0) 112 (59.3) 21 (11.1) 0 (0.0) 0 (0.0) 0 (0.0) 21 (11.1) 21 (11.1) 0 (0.0) 0 (0.0) 91 (48.2) 1 (0.5) 0 (0.0) 1 (0.5) 90 (47.7) 71 (37.6) 19 (10.1) 0 (0.0) 9 (4.8) 9 (4.8) 1 (0.5) 0 (0.0) 1 (0.5) 0 (0.0) 8 (4.3) 6 (3.2) 2 (1.1) 0 (0.0) 3 (1.6) 1 (0.5) 4 (2.1) 0 (0.0) 8 (4.2)  148 (78.3) 3 (1.6) 35 (18.5) 35 (18.5) 0 (0.0) 110 (58.2) 20 (10.6) 0 (0.0) 0 (0.0) 0 (0.0) 20 (10.6) 20 (10.6) 0 (0.0) 0 (0.0) 90 (47.6) 0 (0.0) 0 (0.0) 0 (0.0) 90 (47.6) 69 (36.5) 18 (9.5) 3 (1.6) 10 (5.2) 10 (5.2) 1 (0.5) 0 (0.0) 0 (0.0) 1 (0.5) 9 (4.7) 5 (2.6) 4 (2.1) 0 (0.0) 6 (3.2) 0 (0.0) 12 (6.3) 0 (0.0) 13 (6.9)  143 (75.6) 4 (2.1) 29 (15.3) 29 (15.3) 0 (0.0) 110 (58.2) 21 (11.1) 0 (0.0) 0 (0.0) 0 (0.0) 21 (11.1) 21 (11.1) 0 (0.0) 0 (0.0) 89 (47.1) 0 (0.0) 0 (0.0) 0 (0.0) 89 (47.1) 66 (34.9) 20 (10.6) 3 (1.6) 11 (5.7) 11 (5.7) 2 (1.0) 0 (0.0) 1 (0.5) 1 (0.5) 9 (4.7) 5 (2.6) 4 (2.1) 0 (0.0) 10 (5.3) 0 (0.0) 14 (7.4) 0 (0.0) 11 (5.8)  128 (67.7) 4 (2.1) 25 (13.2) 25 (13.2) 0 (0.0) 99 (52.4) 16 (8.5) 0 (0.0) 0 (0.0) 0 (0.0) 16.5 (8.5) 16 (8.5) 0 (0.0) 0 (0.0) 83 (43.9) 0 (0.0) 0 (0.0) 0 (0.0) 83 (43.9) 64 (33.9) 18 (9.5) 1 (0.5) 15 (7.9) 15 (7.9) 2 (1.0) 0 (0.0) 1 (0.5) 1 (0.5) 13 (6.9) 6 (3.2) 7 (3.7) 0 (0.0) 12 (6.3) 0 (0.0) 20 (10.6) 1 (0.5) 13 (6.9) 30  N(%) FU #1 (n=189) FU #2 (n=189) FU #3 (n=189) FU #4 (n=189) FU #5 (n=189) Facility Descriptors            Staffing level           24 hour staffing           Day staff           Community support only  169 (89.9) 10 (5.3) 8 (4.2)  147 (79.5) 20 (10.8) 15 (8.1)  131 (74.0) 19 (10.7) 21 (11.9)  129 (73.7) 20 (11.4) 20 (11.4)  118 (70.7) 19 (11.4) 21 (12.6)      Staff Ratio           1:1           1:2           1:3 or more  72 (38.3) 89 (47.3) 26 (13.8)  53 (28.6) 76 (41.1) 53 (28.6)  39 (22.0) 69 (39.0) 63 (35.6)  34 (19.5) 68 (39.1) 65 (37.4)  34 (20.5) 53 (31.9) 70 (42.2)      Number of residents           1 - 4           5 - 8           8 - 12           13 – 25           > 26  13 (6.9) 4 (2.1) 8 (4.3) 89 (47.3) 73 (38.8)  16 (8.6) 19 (10.3) 9 (4.9) 75 (40.5) 63 (34.0)  23 (13.0) 23 (13.0) 6 (3.4) 65 (36.7) 54 (30.5)  23 (13.2) 24 (13.8) 6 (3.4) 62 (35.6) 51 (27.0)  23 (13.9) 20 (12.0) 10 (6.0) 51 (30.7) 51 (30.7)      Programming           Available on site           Available off site only  167 (88.8) 20 (10.6)  142 (76.8) 40 (21.6)  125 (70.6) 46 (26.0)  121 (69.1) 48 (27.4)  111 (66.5) 47 (28.1)      Design Characteristics           Purpose Build Mental Health Facility           Converted Mental Health Facility  166 (88.8) 20 (10.7)  146 (78.9) 36 (19.5)  128 (72.3) 43 (24.3)  124 (70.9) 45 (25.7)  112 (67.1) 46 (27.5)      Expected Length of Stay           Long-Term           Short-Term           Variable  10 (5.3) 4 (2.1) 173 (92.0)  21 (11.4) 10 (5.4) 151 (81.6)  27 (15.3) 13 (7.3) 131 (74.0)  30 (17.1) 13 (7.4) 126 (72.0)  36 (21.6) 11 (6.6) 111 (66.5) Facility Descriptive (see figure 1)            Acute/Intensive [n at RVH]      TPRF **       CPRF **       Extended/Intermediate 72 (38.1) [42] 89 (47.1) 10 (5.3) 2 (1.1) 53 (28.0) [35] 75 (39.7) 24 (12.7) 4 (2.1) 37 (19.6) [24] 68 (36.0) 21 (11.1) 8 (4.2) 34 (18.0) [20] 64 (33.9) 20 (10.6) 12 (6.3) 29 (15.3) [15] 52 (27.5) 20 (10.6) 20 (10.6)       Mixed       Semi-independent living       Boarding home       Independent living 0 (0.0) 3 (1.6) 3 (1.6) 8 (4.2) 4 (2.1) 6 (3.2) 2 (1.1) 14 (7.4) 6 (3.2) 6 (3.2) 3 (1.6) 21 (11.1) 5 (2.6) 8 (4.2) 4 (2.1) 19 (10.0) 6 (3.2) 6 (3.2) 3 (1.6) 20 (10.6)       SRO       Forensic hospital       Correctional facility       Homeless shelter       Deceased       Unknown  0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 1 (0.5) 1 (0.5) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 4 (2.1) 3 (1.6) 0 (0.0) 1 (0.5) 0 (0.0) 0 (0.0) 12 (6.3) 6 (3.2) 1 (0.5) 1 (0.5) 0 (0.0) 0 (0.0) 14 (7.4) 7 (3.7) 1 (0.5_ 1 (0.5) 1 (0.5) 0 (0.0) 20 (10.6) 10 (5.3) 31  * Unknown refers to cases in which the location of the participant was known; however there was insufficient data to characterize the facility into independent, semi-independent or boarding homes.   **  SRO:  Single Room Occupancy Hotel; CPRF:  Community Psychiatric Residential Facilities; TPRF:  Tertiary Psychiatric Residential Facilities 32  Figure 2.1  Assessment Facilities – Descriptive    *  Unknown refers to cases in which the location of the participant was known; however there was insufficient data to characterize the facility into independent, semi-independent or boarding homes.   **  SRO:  Single Room Occupancy Hotel; CPRF:  Community Psychiatric Residential Facilities; TPRF:  Tertiary Psychiatric Residential Facilities   33  2.4 Discussion Results from this study confirm findings from numerous research projects in Canada and around the world.  When a program of deinstitutionalization is well planned and well-resourced, negative outcomes such as transinstitutionalization and homelessness can be avoided and positive clinical and psychosocial outcomes can be achieved. We were able to assess the location of all initial study participants at each follow-up for the entire length of the study.  Therefore, we are confident that only one participant spent any time residing in a correctional facility, one participant in a forensic hospital, and only one participant spent a small amount of time homeless.  These results echo findings from a recent administrative data based study which also demonstrated the realignment of psychiatric services in British Columbia did not result in transinstitutionalization (Livingston et al., 2011).  A research study in Quebec found similar results with only two participants requiring care at a forensic hospital, no participants residing in jails or prisons, and two participants lost to follow-up who were suspected to be homeless (Lesage et al., 2000).  Similar results have also been replicated in other countries at the vanguard of deinstitutionalisation. For example the TAPS Project found that in five years of follow-up, four participants became homeless (Leff et al., 2000), and three resided in a prison (Trieman et al, 1998).  In Sum, when best practices for the devolution of psychiatric services are followed, transinstitutionalization and homelessness do not follow. The clinical and psychosocial outcomes demonstrated in this study parallel results from similar studies in industrialized/resource-rich countries. Consistent with findings in Indiana and Australia, we found no change in clinical symptoms and a significant increase in participant perceived quality of life (McGrew et al., 1999; Hobbs et al., 2002).  In addition, participants 34  evidenced decreases in some domains associated with the negative syndrome of schizophrenia including improved independent living skills and even some reduction in cognitive deficits.  These results support recovery models advocating that even long-stay patients can benefit from well executed psychosocial rehabilitation programs (Briand et al., 2006). Approximately 11%of our participants died over the course of the study.  We did not have a control group so we are unable to assess the possibility of excess mortality.  However, given the research showing elevated rates of mortality among individuals living with mental illness in general, and schizophrenia more specifically, as well as the results from this study demonstrating a significant decrease in health maintenance behaviours, this is an important direction to pursue in future research (Lesage et al., 1990; Lawrence et al., 2010; Tiihonen, et al., 2009). The majority of our participants remained in facilities that provided 24 hour care (~70%).  However, a slow movement towards smaller facilities with less onsite programming was evidenced.   A similar trend has been reported elsewhere in the literature.  For example, the TAPS project found that most of their participants required facilities with the same level of support and there was only a slight trend for movement towards less supervised facilities (Trieman et al., 1998). The limitations of this study originate from the realities of studying an evolving redevelopment project, with a severely disabled population, in geographically expansive province.  For example, one cohort of patients who moved to a facility in the far North of British Columbia were only interviewed every second year due to financial realities.  A significant number of participants at each follow-up who were either too ill to be interviewed or refused to be interviewed.  To address these limitations, we conducted analyses to examine possible 35  systematic biases in the missing data and found none modifying the results.   Despite the limitations imposed by the realities of studying a real-life event in real time, the strengths are a 5-year follow-up that traced all initial long-stay inpatients and used standardised questionnaires with patients and staff to demonstrate the effectiveness of a well-planned and resourced deinstitutionalization project. Forthcoming analyses will explore the facility and clinical characteristics that predict both positive and negative outcomes to examine whether resources were used efficiently.  In the future, it will be important to obtain budgetary information from each of the five Health Authorities in order to explore the relationship between cost efficiency and clinical effectiveness. Additionally, one of the paper’s authors has experience in evaluating the cost effectiveness of large-scale psychiatric hospital deinstitutionalizations; we will endeavour to obtain budget information for each of the five Health Authorities in order to conduct cost-benefit analyses.    36  Chapter 3: Forensic and Tertiary Civil Psychiatric Inpatients: Comparing Psychosocial, Clinical, and Risk Profiles, and the Prevalence of Negative Outcomes 3.1 Introduction Researchers, politicians, policy makers and mental health activists alike have suggested that bed downsizing and hospital closures (i.e., deinstutionalization) have led to the transinstitutionalization of severely mentally ill persons from the civil psychiatric system to other social service agencies, in particular, the criminal justice system and the forensic psychiatric system (e.g., Peternelj-Talyor, 2008; Priebe & Turner, 2003). This perspective has drawn support from evidence of the steady increase in demand for forensic mental health services around the world (Jansman-Hart et al., 2011). It is unknown whether or not the increased demand on the forensic system is a result of changes in criminal and/or civil laws, changes in the nature of the population (e.g., patients with more refractory illness, patients with behavioural difficulties, and/or patients with comorbid substance use disorders), the failure to transfer resources and services to the community in order to treat the most difficult to manage patients (Kirby, 2006) and/or increased use of the criminal justice system to manage difficult to treat patients (Jansman-Hart et al., 2011).  Although we have some information regarding similarities and differences in risk factors and service use between severely mentally ill individuals in the prison system versus those in the forensic system (e.g,. Dumais, Côté, Larue, Goulet, & Pelletier, 2014) or severely mentally ill individuals in the prison system versus civil psychiatric patients (e.g., Côté, Lesage, Chawky, & Loyer, 1997);, there remains a dearth of extant literature comparing forensic and civil psychiatric  37  populations, despite the fact that there appears to be considerable overlap in the characteristics of the populations (persistently mentally ill, primarily schizophrenia spectrum disorders) and their needs and risk profiles (e.g., aggression, self-harm, suicide). There have been a couple previous studies that compared and contrasted forensic and civil psychiatric inpatient samples, for example:  Heilbrun et al. (1995) focused primarily on seclusion and restraint; Seto et al. (2004) examined criminogenic, clinical, and social problems of the two groups; Dumont et al. (2012) studied clinical characteristics that acted as obstacles in the transition to community-based care; and Landgraf et al. (2013) compared clinical and demographic characteristics of female forensic and civil psychiatric patients. Results from these studies had very little overlap and reported divergent results.  This study was designed to compare and contrast the psychosocial, clinical, and risk profiles of civil and forensic psychiatric patients in a jurisdiction where these two systems largely function independently (Every-Palmer et al., 2013). In addition, the prevalence of a variety of negative outcomes (i.e. aggression, inappropriate sexual behaviour, self-harm, suicide, self-neglect, unauthorized leave, substance abuse, victimization, and stalking perpetration) were compared over a six month time frame. Understanding the psychosocial and risk profile and the prevalence of negative outcomes of concern in these two populations is essential for informing policy and practice. 3.2 Method This research was approved by the applicable universities and health authorities. 3.2.1 Setting British Columbia (BC), Canada is uniquely situated to investigate the similarities and differences between civil and forensic populations because its forensic and civil psychiatric  38  systems function independently (Every-Palmer et al., 2013). In many jurisdictions these two systems are either interconnected, and/or are housed in the same location and function in an interconnected manner. This interconnection makes independent comparison of the two groups difficult. In contrast, in BC there is only one forensic psychiatric hospital (FPH) serving the entire province.  Individuals are assessed and/or treated at the hospital if they are (a) found Not Criminally Responsible on account of Mental Disorder (NCRMD, equivalent to NCR in the United States), (b) referred for assessment or treatment from jail or prison, or (c) found unfit to stand trial. Until recently, all tertiary civil psychiatric patients in the province were housed in one large-scale facility, independent of the forensic hospital. Over the past several decades, BC undertook a province-wide initiative to redevelop and reorganize tertiary civil psychiatric services. Riverview Hospital (RVH), the single tertiary psychiatric hospital in the province, was downsizing for several decades and ultimately closed in July 2012. Several smaller regionalized community-based facilities were opened to accept tertiary patients with the hopes of providing the same (or better) levels of care, service closer to the patients’ families and support systems, and providing a more home-like atmosphere (BC Mental Health & Addiction Services, 2014). 3.2.2 Sample All RVH patients identified for transfer to the smaller tertiary psychiatric facilities as part of the final cohort of patients taking part in the redevelopment of tertiary psychiatric services in BC were approached for participation (August 2008 – July 2012). All participants provided written informed consent prior to participation. Patients deemed incapable of providing informed consent by their psychiatrists were not included in this study (N = 68). Nearly all patients approached agreed to participate (N = 106, 91.67%, this number does not include six patients who left the hospital before we were able to approach them for consent). Data collection at FPH  39  was entirely file-based and did not require written consent. All patients who had an up-to-date Short-Term Assessment of Risk and Treatability assessment (START, Webster et al., 2009 - described below) on file, as per hospital policy, between January 1, 2009 and December 31, 2011 were included (N = 102). 3.2.3 Measures Short-Term Assessment of Risk and Treatability (START; Webster, Martin, Brink, Nicholls, & Desmarais, 2004, 2009). START is a concise clinical guide for the dynamic assessment of seven adverse patient safety outcomes (i.e., violence, suicide, self-harm, self-neglect, unauthorized absence, substance use, and victimization). START guides clinicians toward an integrated, balanced opinion to evaluate the patient’s risk over the short-term (weeks to months). The 20 items are scored concurrently as both vulnerabilities and as strengths, from 0 (not relevant) to 2 (definitely relevant). Each of the seven risk domains are subsequently assessed as low, moderate, or high risk. Total scores for both strengths and vulnerabilities can be calculated for research purposes. START has been found to have good internal consistency (alpha > 0.80), appropriate item homogeneity (MIC = 0.20 - 0.50), good to excellent inter-rater reliability, and good predictive validity (ICC > 0.80) (Nicholls et al., 2006; Wilson et al., 2010).   START Outcome Scale (SOS; Nicholls, Gagnon, Crocker, Brink, Desmarais, & Webster, 2007).  The Overt Aggression Scale (OAS-M; Yudofsky et al., 1986) was designed to assess observable aggressive or violent behavior, as opposed to violent tendencies.  It consists of four categories: verbal aggression, physical aggression against objects, physical aggression against self, and physical aggression against other people. Each category is rated on a 4-point scale from least (1) to most (4) severe. In 2007, OAS-M was revised for use with the START in order to track the prevalence of negative outcomes commonly of concern in institutionalized, mental  40  health, and justice involved populations.  Modifications were intended to capture the START risk domains that were not included on the OAS-M (such as unauthorized leave, substance use, victimization, etc.) and to separate out the self-harm and suicide items. Each adverse outcome category is rated according to its severity on a 4-point scale from least severe (1) to most severe (4), for example, aggression against others is coded from level one: makes threatening gestures, swings at people, grabs at clothing, throws objects dangerously, through to level four: attacks others, uses weapons, resulting in severe physical injury (e.g., fracture, loss of teeth or consciousness, lacerations, internal injury). Prior research shows that the START outcome scale can be reliably coded from patient files (intraclass correlation coefficient = .70; Nicholls et al., 2006; also see O'Shea, Picchioni, & Dickens, 2014). 3.2.4 Procedures At baseline, research assistants with a minimum of a BA in a relevant discipline collected demographic, clinical, and contextual information from files and used this information to complete the START. All research assistants were trained by an author of the START. Six months of follow-up data was collected by a researcher blind to the baseline START data. For the civil psychiatric inpatients, baseline data was collected at RVH and follow-up data was collected at the various smaller scale tertiary facilities scattered across the province. For the forensic inpatients, all data was collected from clinical files at FPH. 3.2.5 Analysis Data analysis was conducted using SPSS (version 16.0). Descriptive comparisons were computed with independent samples t-tests and chi-square tests. If participants were missing five or fewer START strength or vulnerability items, the total score was prorated as per the manual guidelines (Webster et al., 2009).  41  3.3 Results 3.3.1 Psychosocial Characteristics The majority of the forensic sample was male (91.2%), in contrast the civil sample was more gender balanced (61.3% male) (see Table 1). The mean age for the civil psychiatric inpatients was 47 years, significantly older in comparison to the forensic psychiatric inpatients that had a mean age of 40 years (t = 3.56, p < 0.001). The majority of both samples were Caucasian and the two groups had similar educational attainment. The civil inpatients were more likely than the forensic inpatients to be married or previously married. Differences in legal status were a function of the way in which patients entered the system; civil patients were most often designated as involuntary and forensic patients most often designated as NCRMD. The predominant primary diagnosis for both groups was in the schizophrenia spectrum. Forensic patients were more likely than civil patients to have a dual diagnosis involving a substance use disorder. The age at which participants were first hospitalized in a psychiatric facility was nearly identical; however, the civil psychiatric patients had been hospitalized significantly more often. The majority of both groups had an index hospitalization of longer than one year; however, the civil psychiatric cohort had been hospitalized for significantly longer during the index admission than the forensic cohort (see Table 1).    42  Table 3.1  Psychosocial Characteristics at Baseline Characteristic Civil Inpatients (N=106) Forensic Inpatients (N=102) p-Value Gender      Male      Female  65 (61.3%) 41 (38.7%)  93 (91.2%) 9 (8.8%)  <0.001 Age 47.04 40.47 <0.001 Ethnicity      Caucasian      First Nations      Asian      Other  80 (75.5%) 8 (7.5%) 9 (8.5%) 9 (8.5%)  75 (73.5%) 5 (4.9%) 8 (7.9%) 14 (13.7%)  0.59  Marital Status      Single      Divorced      Common Law      Married      Separated      Widowed      Other  74 (70.4%) 18 (17.1%) 3 (2.9%) 5 (4.8%) 2 (1.9%) 2 (1.9%) 1 (1.0%)  84 (84.0%) 5 (5.0%) 0 (0.0%) 6 (6.0%) 3 (3.0%) 2 (2.0%) 0 (0.0%)  0.04 Education      8th Grade or Less      9th- 11th Grade      High School Graduation or GED      Technical or Trade Certificate      Some College/University      Diploma/Bachelor Degree      Masters or PhD  11 (10.6%) 44 (42.3%) 30 (28.8%) 1 (1.0%) 11 (10.6%) 6 (5.7%) 1 (1.0%)  16 (16.3%) 37 (37.8%) 19 (19.4%) 4 (4.1%) 12 (12.2%) 10 (10.2%) 0 (0.0%)  0.26 Legal Status      NCRMD      Remand      Unfit      Involuntary      Voluntary  NA NA NA 100 (95.2%) 5 (4.8%)  92 (90.2%) 2 (2.0%) 3 (2.9%) 5 (4.9%) NA  <0.001 Primary Diagnosis      Schizophrenia Spectrum      Substance Induced Psychosis      Mood Disorder      Mental Disorder due to Medical Condition      Cognitive Impairment      Mental Retardation      Other      No Diagnosis  94 (89.5%) 0 (0.0%) 5 (4.8%) 0 (0.0%) 5 (4.8%) 0 (0.0%) 1 (0.9%) 0 (0.0%)  82 (80.4%) 3 (2.9%) 9 (8.8%) 3 (2.9%) 1 (1.0%) 1 (1.0%) 2 (2.0%) 1 (1.0%)  0.07 Dual Diagnosis      Substance Use      Personality Disorder      Cognitive Impairment  35 (33.0%) 33 (31.1%) 36 (34.0%)  62 (61.0%) 41 (40.2%) 25 (24.5%)  <0.001 0.17 0.13 Age of First Psychiatric Hospitalization 23.97 (10.84) 22.69 (9.51) 0.40  43  Characteristic Civil Inpatients (N=106) Forensic Inpatients (N=102) p-Value Number of Prior Psychiatric Hospitalizations      None      At Least 1      At Least 2      More than 3  0 (0.0%) 7 (6.9%) 7 (6.9%) 87 (86.2%)  13 (12.7%) 21 (20.6%) 16 (15.7%) 52 (51.0%)  <0.001 Length of Index Hospitalization      Less than 1 Year      Between 1 and 2 Years      Between 2 and 5 Years      Between 5 and 10 Years      More than 10 Years  22 (21.0%) 17 (16.2%) 18 (17.1%) 26 (24.7%) 22 (21.0%)  35 (34.3%) 17 (16.7%) 26 (25.5%) 13 (12.7%) 11 (10.8%)  0.02  3.3.2 Historical Risk Factors Compared to the civil psychiatric inpatients, the forensic inpatients were more likely to have a history of violence and substance abuse. The forensic inpatients also had more extensive criminal histories than the civil patients. Forensic inpatients had more convictions as an adult and were more likely to have been convicted of a violent crime than their civil counterparts. It is important to note that although the forensic inpatients were significantly more likely to have had a history of violence (98%), prior violence was also very common among civil inpatients (81.1%) (see Table 2). There were no differences between civil and forensic inpatients on several key historic risk factors: criminal charges as an adult; being incarcerated as an adult; prior hospitalization in a forensic hospital; being previously found NCRMD; and having been previously been found unfit to stand trial (see Table 2). There were also no differences evident between the two samples with regard to histories of self-harm, suicidality, unauthorized leave, self-neglect, and victimization. Table 3.2  Historical Risk Factors at Baseline Characteristic Civil Inpatients (N=106) Forensic Inpatients N=(102) p-Value Charged as an Adult 53 (53.0%) 63 (61.8%) 0.21 Convicted as an Adult 38 (38.4%) 52 (52.0%) 0.05  44  Characteristic Civil Inpatients (N=106) Forensic Inpatients N=(102) p-Value Number of Convictions      None      At Least 1      At Least 2      More than 3  61 (62.2%) 19 (19.4%) 6 (6.2%) 12 (12.2%)  48 (48.0%) 12 (12.0%) 7 (7.0%) 33 (33.0%)  0.01 Convicted of a Violent Crime as an Adult 23 (24.5%) 39 (39.4%) 0.03 Incarcerated as an Adult 25 (25.8%) 36 (36.7%) 0.09 Number of Incarcerations      None      At Least 1      At Least 2      More than 3  72 (74.2%) 16 (16.5%) 1 (1.1%) 8 (8.2%)  62 (66.0%) 11 (11.7%) 5 (5.3%) 16 (17.0%)  0.01 Forensic Hospitalization  33 (32.0%) 37 (37.0%) 0.46 Previously Found NCRMD *  10 (10.5%) 8 (8.0%) 0.54 Previously Found Unfit to Stand Trial 7 (7.7%) 13 (13.1%) 0.22 History of Violence 87 (81.1%) 100 (98.0%) 0.00 History of Self-Harm 29 (27.4%) 32 (31.4%) 0.53 History of Suicidality  49 (46.2%) 53 (52.0%) 0.41 History of Unauthorized Leave 64 (59.4%) 54 (52.9%) 0.35 History of Substance Abuse 68 (64.2%) 81 (79.4%) 0.02 History of Self Neglect 66 (62.3%) 56 (54.9%) 0.28 History of Victimization as an Adult 54 (50.9%) 42 (41.2%) 0.16 *Note. Not Criminally Responsible on account of Mental Disorder. 3.3.3 START Risk Profile There were no significant differences between civil and forensic inpatients in terms of START total strength scores and total vulnerability scores (see Table 3). The range for both strength and vulnerability scores was quite comparable. Forensic inpatients were significantly less likely than civil inpatients to have a staff member with whom they had a therapeutic alliance or to have a positive peer in their support network (i.e., a friend/co-patient who provides socially appropriate support; a ‘peer’ does not include staff, professionals, family members). START assessments indicated that the estimated risk of harm to self or others and substance misuse was comparable between the civil and forensic inpatients. However, the civil psychiatric inpatients were considered to present a greater risk of taking unauthorized leave, neglecting their self-care  45  and/or being victimized (victimization was defined broadly to include physical, sexual, financial, etc).   Table 3.3  START Risk Profile Characteristic Civil Inpatients (N=106) Forensic Inpatients (N=102) p-Value Strength Total Score      Range      Mean      Std. Deviation  0.00- 32.00 13.27 6.49  0.00 – 38.00 13.37 7.59   0.92 Vulnerability Total Score      Range      Mean      Std. Deviation  3.00 – 34.00 19.30 6.35  2.00 – 33.00 17.63 7.94   0.09 Therapeutic Alliance Present      Yes  64 (60.4%)  36 (35.3%)  0.00 Positive Peer Support Present     Yes  55 (51.9%)  13 (12.7%)  <0.001 Violence Risk Estimate      Low      Moderate      High  71 (67.0%) 24 (22.6%) 11 (10.4%)  61 (59.8%) 31 (30.4%) 10 (9.8%)  0.45 Self-Harm Risk Estimate      Low      Moderate      High  97 (91.5%) 7 (6.6%) 2 (1.9%)  97 (95.1%) 3 (2.9%) 2 (2.0%)  0.47 Suicide Risk Estimate      Low      Moderate      High  101 (95.3%) 4 (3.8%) 1 (0.9%)  98 (96.1%) 4 (3.9%) 0 (0.0%)  0.62 Unauthorized Leave Risk Estimate      Low      Moderate      High  64 (60.4%) 26 (24.5%) 16 (15.1%)  87 (85.3%) 15 (14.7%) 0 (0.0%)  <0.001 Substance Misuse Risk Estimate      Low       Moderate      High  75 (70.8%) 20 (18.8%) 11 (10.4%)  63 (61.7%) 28 (27.5%) 11 (10.8%)  0.32 Self Neglect Risk Estimate      Low      Moderate      High  38 (35.8%) 40 (37.7%) 28 (26.5%)  66 (64.7%) 31 (30.4%) 5 (4.9%)  <0.001 Victimization Risk Estimate      Low      Moderate      High  65 (61.9%) 37 (35.2%) 3 (2.9%)  83 (81.4%) 12 (11.8%) 7 (6.8%)  <0.001  46  3.3.4 Prevalence of Adverse Outcomes Six Months Following the Completion of START Assessments When we collapsed across severity categories to obtain a general picture of the prevalence of adverse events among the two groups a very clear distinction emerged. Across all negative outcomes examined, with the exception of stalking behaviours and self-harm (which were relatively uncommon and were not evident at the most serious level), civil psychiatric inpatients were as likely (aggression against property, suicidal behaviour, substance abuse) or more likely (verbal aggression, aggression against others, sexually inappropriate, suicidal ideation, self-neglect, unauthorized leave, victimization) to have experienced the adverse events we studied (see Table 4).   Significantly fewer civil (2.8%) than forensic (11.8%) inpatients did not experience any of the adverse outcomes measured during the six month follow-up period (Fisher’s Exact Test p=0.02). In contrast, significantly more civil (68.9%) than forensic (38.2%) inpatients experienced at least one serious adverse event (defined as level three or four severity on the SOS) (Table 5). Of interest, there was a very low prevalence of serious incidents across diverse negative outcomes for both civil and forensic inpatients. For both cohorts, the frequency of severe (level 3 or 4) inappropriate sexual behaviour, self-harm, suicidal ideation, suicidal behaviour, substance abuse and stalking were all less than 10% (see Table 5). That being said, serious adverse events were reported for two-thirds of civil patients (68.9%) and one third of forensic patients (38.2%) in this reasonably short follow-up time-frame. Civil inpatients were significantly more likely than forensic inpatients to experience severe self-neglect, unauthorized leave, substance abuse, and victimization (see Table 5).  47  Table 3.4  Negative Outcomes:  Prevalence and Nature of Adverse Incidents in the Six Months Following the Completion of START Assessments Outcome Civil Inpatients (N/% Yes) Forensic Inpatients (N/% Yes) p-Value (chi-square) Verbal Aggression      Any      Level One      Level Two      Level Three      Level Four  82 (77.4%) 65 (61.9%) 34 (32.1%) 23 (21.7%) 0 (0.0%)  56 (54.9%) 54 (52.9%) 28 (27.5%) 22 (21.6%) 5 (4.9%)  <0.001  Aggression Against Property      Any      Level One      Level Two      Level Three      Level Four  43 (40.6%) 27 (26.5%) 14 (13.2%) 4 (3.8%) 0 (0.0%)  30 (29.4%) 24 (22.9%) 17 (16.7%) 10 (9.8%) 0 (0.0%)  0.09  Aggression Against Others       Any      Level One      Level Two      Level Three      Level Four  54 (50.9%) 30 (28.3%) 28 (26.4%) 11 (10.4%) 0 (0.0%)  36 (35.3%) 25 (24.5%) 20 (19.6%) 14 (13.7%) 0 (0.0%)  0.02  Inappropriate Sexual Behaviour       Any      Level One      Level Two      Level Three      Level Four  29 (27.4%) 21 (19.8%) 4 (3.8%) 6 (5.7%) 0 (0.0%)  15 (14.7%) 14 (13.7%) 5 (4.9%) 1 (1.0%) 0 (0.0%)  0.02  Self-Harm       Any      Level One      Level Two      Level Three      Level Four  13 (12.3%) 8 (7.5%) 8 (7.5%) 1 (0.9%) 0 (0.0%)  14 (13.7%) 10 (9.8%) 7 (6.9%) 4 (3.9%) 0 (0.0%)  0.75  Suicidal Ideation      Any      Level One      Level Two      Level Three      Level Four  25 (23.6%) 17 (16.2%) 6 (5.7%) 2 (1.9%) 0 (0.0%)  8 (7.8%) 8 (7.9%) 2 (2.0%) 2 (2.0%) 0 (0.0%)  <0.001  Suicidal Behaviour       Any      Level One      Level Two      Level Three      Level Four  7 (6.6%) 2 (1.9%) 2 (1.9%) 0 (0.0%) 0 (0.0%)  4 (3.9%) 3 (2.9%) 2 (2.0%) 0 (0.0%) 0 (0.0%)  0.54  Self-Neglect      48  Outcome Civil Inpatients (N/% Yes) Forensic Inpatients (N/% Yes) p-Value (chi-square)      Any      Level One      Level Two      Level Three      Level Four 86 (81.1%) 62 (60.8%) 21 (19.8%) 7 (6.7%) 2 (1.9%) 64 (62.7%) 61 (64.9%) 7 (7.0%) 2 (2.0%) 0 (0.0%) <0.001  Unauthorized Leave       Any      Level One      Level Two      Level Three      Level Four  60 (56.6%) 33 (31.1%) 4 (3.8%) 12 (11.3%) 3 (2.8%)  14 (13.7%) 11 (10.8%) 0 (0.0%) 2 (2.0%) 1 (1.0%)  <0.001  Substance Abuse      Any      Level One      Level Two      Level Three      Level Four  26 (24.5%) 13 (12.3%) 4 (3.8%) 2 (1.9%) 0 (0.0%)  25 (24.5%) 23 (22.5%) 10 (9.8%) 0 (0.0%) 0 (0.0%)  1.00  Victimization       Any      Level One      Level Two      Level Three      Level Four  60 (56.6%) 26 (24.5%) 9 (8.5%) 11 (10.4%) 1 (0.9%)  36 (35.3%) 22 (21.6%) 12 (11.8%) 10 (9.8%) 1 (1.0%)  <0.001  Stalking       Any      Level One      Level Two      Level Three      Level Four  6 (5.7%) 4 (3.8%) 1 (0.9%) 0 (0.0%) 0 (0.0%)  15 (14.7%) 11 (10.8%) 6 (5.9%) 0 (0.0%) 0 (0.0%)  0.03    Table 3.5  Prevalence of Severe Adverse Events in the Six Months Following the Completion of START Assessments Serious Adverse Events* Civil Inpatients (N/% Yes) Forensic Inpatients (N/% Yes) p-Value (chi-square/Fisher’s Exact Test) Any Serious Adverse Outcome 79 (68.9%) 39 (38.2%) <0.001 Verbal Aggression 32 (30.2%) 26 (25.5%) 0.45 Aggression Against Property 8 (7.5%) 10 (9.8%) 0.56 Aggression Against Others  17 (16.0%) 14 (13.7%) 0.64 Inappropriate Sexual Behaviour  7 (6.6%) 1 (1.0%) 0.07 Self-Harm  1 (0.9%) 4 (3.9%) 0.21  49  Serious Adverse Events* Civil Inpatients (N/% Yes) Forensic Inpatients (N/% Yes) p-Value (chi-square/Fisher’s Exact Test) Suicidal Ideation 2 (1.9%) 2 (2.0%) 1.00 Suicidal Behaviour  1 (0.9%) 0 (0.0%) 1.00 Self-Neglect 17 (16.0%) 2 (2.0%) <0.001 Unauthorized Leave  29 (27.4%) 3 (2.9%) <0.001 Substance Abuse 7 (6.6%) 0 (0.0%) 0.01 Victimization  22 (20.8%) 10 (9.8%) 0.03 Stalking  0 (0.0%) 0 (0.0%) -- *Note. ‘Serious’ refers to collapsed categories from Levels 3 and 4 of the START Outcome Scale (SOS)  3.4 Discussion The current study found significant differences between civil and forensic psychiatric inpatients on a number of static, historical, and criminogenic factors. A preliminary examination of the presenting profile of the forensic inpatients in this study would suggest that they are a higher risk cohort than the civil inpatients. Although there were many similarities, as would be expected, the forensic inpatients clearly presented with much more pronounced criminal histories than the civil inpatients; whereas the civil inpatients tended to present with more persistent and debilitating mental disorder, as indicated by a greater number of hospitalizations, more lengthy index admissions and greater prevalence and severity of self-neglect. However, a more thorough examination of the patients’ risks and needs using comprehensive START assessments suggested the two groups were comparable in many respects. In contrast to the static factors, the START risk profiles of the two groups were remarkably similar. For example, there were no significant differences between the two cohorts on four of the seven risk estimates and on START strength and vulnerability total scores. Despite the historical risk profiles of the forensic inpatients, and consistent with the START assessments, the civil inpatients evidenced a similar or greater  50  prevalence of many negative outcomes, including: verbal aggression, aggression against others, inappropriate sexual behaviour, suicidal ideation, self-neglect, unauthorized leave, and victimization. In addition, significantly fewer civil psychiatric inpatients experienced an absence of any negative outcomes during the six month follow-up period. Of note, severe (level three or four on the SOS) adverse events were relatively infrequent among both samples; nevertheless, civil inpatients were significantly more likely than forensic inpatients to experience severe self-neglect, unauthorized leave, substance abuse, and victimization. Examples of severe outcomes in these categories include:  self-neglect level three – unsafe behavior likely to result in serious implications (e.g. not following up on medical testing); unauthorized leave level three – absconds from escorted leave or is returned by police from unescorted leave or does not return; substance abuse level 3 – frequent substance use leading to significant physical, behavioural, emotional, relationship, occupational, or educational impairment; and victimization level three – physical assault resulting in mild-moderate physical injury (e.g., bruises, sprains, or welts) (SOS, Nicholls, et al., 2007). Forensic inpatients were found to be significantly less likely than civil inpatients to have a therapeutic alliance with a caregiver or have a positive peer in their support network providing support. These finding have important implications for risk management and treatment planning. For example, patients who experience a therapeutic alliance have been found to demonstrate a more positive course while in hospital and better outcomes post-hospitalization (Hewitt & Coffey, 2005).   The implications of the findings need to be considered with due consideration to the study limitations. The most significant limitation of this study relates to sampling methodology. The forensic sample included all current inpatients with an up-to-date START on file during the study time frame. Completing STARTs within one month of admission and every three months  51  thereafter for all treatment patients is policy at the forensic hospital. Audited compliance rates indicate that during the study period 80-90% of patients had up-to-date STARTs on file. In contrast, for the civil cohort, those inpatients whose psychiatrist deemed too ill to participate in the study were excluded. This means that some of the most difficult to manage civil inpatients were not included, indicating that our results may well underestimate the prevalence and severity of negative outcomes among tertiary civil psychiatric inpatients. Additionally, we collected information on the prevalence of outcomes from multidisciplinary progress notes. It is possible that care providers recorded adverse events differently from one setting to another and/or that low-severity events are not coded reliably (see Ehmann et al., 2001). 3.5 Conclusions This study provides important insights into the similarities and differences between civil and forensic inpatients in terms of static risk factors, short-term dynamic risk profiles, and the prevalence of a diverse range of negative outcomes. Specifically, despite the challenging static risk profile of forensic inpatients, there were few differences in dynamic risk as assessed by START. Consistent with prior research (e.g., Wilson et al., 2013), the results lend support for the value and importance of integrating dynamic variables into risk assessments and treatment and management planning, rather than relying on static and historical information. Civil psychiatric patients experienced many of the negative outcomes in a greater variety and severity than the forensic psychiatric patients. These findings demonstrate the extent to which risk assessment and treatment planning needs in civil psychiatry overlap with current practices in the forensic setting and highlight the importance of ensuring that the expertise in violence risk assessment and risk management commonly of focus in forensic services is also integrated into civil psychiatric care practices. The projection of forensic expertise ‘upstream’ into civil psychiatric training programs  52  and treatment settings may provide a means of preventing adverse events that may result in serious harm to the patient, the public and/or the criminalization of mental illness (e.g., Crocker et al., 2015).   53  Chapter 4:   A Clinical and Risk Profile of Forensic Psychiatric Patients:  Treatment Team STARTs in a Canadian Service2  4.1 Introduction Best Practice in Managing Risk (Department of Health, 2007) details 16 best practice guidelines for effective clinical risk assessment and risk management within mental health services. The guidelines endorse the structured professional judgment (SPJ) approach to risk assessment and risk management.  They further emphasize evidence-based practice and collaboration with the service user as crucial components in the process of decision-making. The Department of Health (DoH) document also recommends multidisciplinary work-groups, well-considered and well-delivered training, and clear procedures on how, and to whom, risks must be communicated.  Finally, the recommendations encouraged ‘self-reflective’ practice and updated training every three years, at a minimum.  Consistent with this approach, the American Psychological Association’s 2005 Presidential Task Force on Evidence-Based Practice (American  Psychological Association,  2006) also recently concluded that it is important to “assess patient pathology as well as clinically-relevant strengths” (p. 276) when evaluating and treating clients. Finally, the Royal College of Psychiatrists, London, (2008) convened a scoping group of national and internationals experts that similarly concluded risk assessment and management should acknowledge and promote client’s strengths and support recovery.  Both                                                  2 Nicholls, T., Petersen, K. L., Brink, J., & Webster, C. (2011). A clinical and risk profile of forensic psychiatric patients: Treatment team STARTs in a Canadian service. The International Journal Of Forensic Mental Health, 10(3), 187-199. doi:10.1080/14999013.2011.600234  54  organizations and individual practitioners can gauge their current practice against these guidelines (Haque, Cree, Webster, & Hasnie, 2008). As this special issue demonstrates, until recently there has been a clear and overwhelming tendency to focus on risks, deficits, or vulnerabilities in the risk assessment field (Rogers, 2000) with few SPJ guides available to help clinicians’ construct positive treatment plans and to ensure that their assessments address these ethical and professional recommendations.  The Short-Term Assessment of Risk and Treatability (START; Webster, Martin, Brink, Nicholls, & Middleton, 2004; Webster, Martin, Brink, Nicholls, & Desmarais, 2009), and the Structured Assessment of Protective Factors for Violence Risk (SAPROF; de Vogel, de Ruiter, Bouman, & de Vries Robbé, 2009) are two of the few measures that satisfy the requirement to address clients’ strengths as well as their deficits (also see Structured Assessment of Violence Risk in Youth (SAVRY) Borum, Bartel, & Forth, 2006;  Inventory of Offender Risk, Needs, and Strengths (IORNS), Miller, 2006).  4.1.1 Short-Term Assessment of Risk and Treatability (START) The START is a concise clinical guide for the assessment and management of short-term risks (weeks to months) (for a brief explanation see Nicholls, Webster, Nicholls, Brink, & Martin, Desmarais, & Brink, 20062008; Webster, Nicholls, Martin, Desmarais, & Brink, 2006;see Appendix 1).  The scheme is unique relative to existing violence risk instruments because it guides clinicians to consider historical markers, as well as both vulnerabilities and strengths, simultaneously, on 20 dynamic items.  START is intended to inform clinical interventions and assist in treatment and risk management efforts aimed at averting the types of adverse events that occur at elevated rates in populations of mentally and personality disordered  55  individuals (violence, self-harm, suicide, substance abuse, unauthorized leave, self-neglect, and being victimized).  The measure was developed with the intention of application to evaluations and care planning with persons living with mental illness and personality disorders in diverse settings (inpatient and community), and across populations (corrections, forensic and civil psychiatry).   4.1.2 International Uptake of the START START has been taken up into clinical practice with considerable enthusiasm.  We are aware of implementations, often supported by well-developed programs of research, in more than 10 countries (Sweden, Australia, United Kingdom, Ireland, Scotland, Denmark, the United States, Finland, South Africa, China, The Netherlands, New Zealand, Norway, Germany, and Singapore).  The rapid integration into clinical practice reflects the efforts of several of our colleagues to make the manual available in diverse languages.  It has now been translated into four languages and, to our knowledge, four additional translations are underway).  This speaks to broad issues in the forensic mental health field, such as the scholarly partnerships the International Association of Forensic Mental Health Services have fostered through our annual conference, as well as our ability to borrow considerably from the success of measures like the HCR-20 (Webster, Douglas, Eaves, & Hart, 1997).  Despite being a relatively new measure, the literature on the START is beginning to grow.    4.1.3 Research on the START 4.1.3.1 User Satisfaction Doyle, Lewis, and Brisbane (2008) evaluated the practical utility and face validity of the  56  START among forensic mental health nursing staff in Manchester, UK.  The majority of respondents reported that the measure assisted in their individual risk formulations, with 85% endorsing ratings from moderate to very useful.  A sizable majority (74.4%) reported no difficulties in completing the Risk Formulation component of the START, and almost a full 95% were at least moderately-to-very confident in rating the START items overall.  Moreover, the process of completing a START was found to be helpful for organizing information, and this was noted as a salient theme emerging from participants’ questionnaire responses.   Desmarais, Collins, Nicholls, and Brink (2011) gauged user-satisfaction on the START via surveys administered to clinical staff at the British Columbia Forensic Psychiatric Hospital (FPH).  Regardless of rater profession, clinicians endorsed the user friendliness items at levels of 80% or higher, with the exception of only one item (finer scoring distinctions: 56%).  A high level of endorsement was reported concerning the measure’s clinical utility; specifically, 93% of respondents agreed that START is a clinically useful tool.  More specifically, 81% of direct-care providers found the inclusion of dynamic items particularly useful, and 93% appreciated the inclusion of both strengths and vulnerabilities.  Overall, the majority of respondents supported the usefulness of the START to inform their clinical practice.   4.1.3.2 Implementation Findings Kroppan, Nesset, Pedersen, Almvik, and Palmstierna (2011) examined the implementation of START in a forensic high secure unit in Norway.  They found that 73% of staff agreed that START contributed significantly to a enhancing a more systematic approach to risk assessment and risk management.  An additional 79% of direct-care providers agreed that the START was useful as a tool in risk assessments and treatment planning.  During interviews,  57  the Norwegian staff members further described how the START structured their clinical judgement and also fulfilled their wish to address both the patients’ strengths and risks.  The START continued to be in regular use after the more intensive implementation process.  The authors concluded that the implementation of START contributed to interdisciplinary security decisions.  As well, as it helped create a broader appreciation of the patients’ circumstances than was the case prior to the implementation of START.   From a study conducted in Quebec, Canada, Crocker, Braithwaite, Laferriere, Gagnon, Venegas, and Jenkins (2011) reported the results of a longitudinal prospective mixed method (qualitative and quantitative) implementation study indicating.  Their results indicated that the START was well-integrated into the unit’s clinical and administrative activities.  They found that, despite being implemented at a much later date than other measures, the uptake of START appeared to be stronger.   4.1.3.3 Validation Studies Nicholls and colleagues (2006) evaluated START assessments completed by nurses, social workers, and psychiatrists in a sample of 137 male forensic psychiatric patients.  The study indicated excellent interrater agreement overall, intraclass correlation coefficient (ICC) = .87, as well as when it was examined by the profession of the evaluator (ICCs for nursing = .88, social work = .92, and psychiatry = .80, all p < .001).  Significantly higher START total scores3                                                 3 Note that at the time of the original validation study, START items were rated on one continuous 6-point scale, from 0 (indicating considerable strength) to 5 (indicating considerable vulnerability) and included final risk estimates for only four of the now seven outcome domains (violence to others, self-harm, suicide, and unauthorized absence).  were reported for patients who engaged in aggression over the 12-month follow-up: any  58  aggression to others (M = 75.66 versus 65.86), verbal aggression (M = 75.86 versus 66.82), aggression against objects (M = 77.90 versus 68.00), physical aggression against others (M = 76.32 versus 68.25), violence against others (M = 81.82 versus 69.12), and sexual aggression (M = 80.63 versus 70.24, all p < .05).  Receiver Operating Characteristic (ROC) analyses of a subsample of 50 patients who remained hospitalized throughout follow-up revealed good validity in predicting both verbal and physical aggression, all in the inpatient setting Area Under the Curve (AUC) > .67, p < .05.  Brathwaite, Charette, Crocker, and Reyes (2010) completed a longitudinal prospective mixed-method study to evaluate whether the START was associated with adverse events that occurred over the short-term (30 days).  START assessments (N = 133) were completed for severely mentally ill patients on an acute civil psychiatric inpatient unit.  A START assessment was also completed on a subset of patients (N = 34) who were on the unit continuously for 30 days prior to and 30 days after the START was completed.  Dependent variables were coded for 12 months using the START Outcome Scale (SOS, Nicholls, Gagnon, Crocker, Brink, Desmarais, & Webster, 2007).  Brathwaite et al. reported that the vulnerability scale significantly predicted physical aggression against others (AUC = .66, p < .05).   A recent analysis of assessments completed on 30 male forensic psychiatric inpatients provided evidence for the ability of the START Version 1.11 to predict short-term violence risk (Wilson et al., 2010).  Specifically, results demonstrated that Strength and Vulnerability total scores as well as the final risk estimates predicted aggressive behavior in the three months following the assessment (AUC > .73, all p < .001).  However, given the small sample size, more research is needed to establish the reliability and validity of START in its current form.  Finally, Desmarais, Nicholls, Wilson, and Brink (2011) examined the predictive and  59  incremental validity of research assistant completed START assessments for 120 male forensic psychiatric patients.  STARTs were completed based on file reviews and outcome data were coded from hospital files for a 12-month follow-up period using the Overt Aggression Scale (OAS).  START evidenced excellent interrater reliability and demonstrated both predictive and incremental validity over the HCR-20 historical factors and the PCL:SV.  Overall, results support the reliability and validity of START and the value of using dynamic risk and protective factors to assess violence risk.  Research on the START, and for that matter most structured professional judgment measures, has focused to a large extent on assessments completed by research assistants (e.g., Desmarais et al., 2011; Wilson et al., 2011).  Although the START has now been implemented across several large mental health systems (e.g., Oregon State Hospital, USA; St. Andrews Healthcare, Northampton, UK) including the British Columbia Forensic Psychiatric Service in Canada, little effort has so far been made to explore the use of the scheme in clinical practice.  To date, research has focused to a large degree on the predictive accuracy of START and has neglected the success of clinical implementations (or field reliability).  In addition to providing valuable data about the START it may well be that insights are to be gained by understanding the clinical profiles of forensic clients when assessed by the START.   4.1.4 The Present Study We had three specific research questions: (1) What is the clinical and risk profile of a large Canadian forensic psychiatric cohort? (2) What are the psychometric properties of the START in a forensic service when the evaluations are completed by clinical teams?  (3) How are these profiles and START outcomes similar or different among male and female patients and  60  across setting (i.e., hospital versus clinic) or security levels (i.e., maximum, medium, minimum)?    4.2 Method 4.2.1 Sample We examined all START forms completed by treatment teams working in a Canadian forensic service over a one-year period (September 1, 2009 to August 31, 2010).  All patients were under the authority of the Forensic Psychiatric Services Commission (FPSC).  We removed subjects for whom too many strength (n = 4) or vulnerability (n = 10) item ratings were missing, to allow us to prorate the scores.  We further removed one trans-gendered patient from the sample to allow us to examine gender similarities and differences.  This resulted in a total of 1057 START forms being available for analyses.    4.2.2 Clinical and Risk Profile of the Sample  The average age of the patients was 39 years (SD = 12.62) for men and 40 years (SD = 11.94) for women.  One third of the sample (men = 26%; women = 23%) were individuals who had been found Not Criminally Responsible on Account of Mental Disorder (NCR-MD (Canada’s insanity defence) and a large proportion of the participants were probationers (men = 26%; women = 34%).  In total, 763 forms were for community client clinics and 292 were from hospital inpatients (n = 2, location unspecified) who represented patients from all three security levels (see Table 1).  Most participants in the present study were diagnosed with schizophrenia spectrum disorders, though these diagnoses were significantly more prevalent among men (54% than women (35%, p < .001).  Chi-squared analyses also indicated that mood disorders were much more common among the women (25%) than among the men (14%, p < .001).  Few other  61  differences were noted between the male and female participants with regard to diagnosis, with the exception that women are more likely than men to be diagnosed with anxiety disorders (8% versus 3%, p = <.001) and personality disorders ( 2% versus 7%, p = <.001) (see Table 4.1).   Not surprisingly, the START assessments suggest that many of the patients have a history of the types of risks and safety concerns denoted on the START (i.e., violence, self-neglect, etc.; see Risk Estimates).  In particular, 83% of the men and 77% of the women have a history of violence (χ2 (1, N = 1057) = 3.55, p < 0.059).  Similarly, the rates of suicide histories (e.g., ideation, attempts) were high in this sample of inpatients: 23% of men and 30% of women (χ2 (1, N = 1057) = 2.46, p = 0.117).  The rates of self-neglect (30% men; 31% women; (χ2 (1, N = 1057) = .010, p = 0.920) and taking unauthorized leave (e.g., not returning from an unescorted absence; wandering off from an escorted outing) were highly comparable across the two genders (22% men; 19% women; (χ2 (1, N = 1057) = 0.559, p =0.455)).  As would be expected, we found very high rates of substance abuse histories among the men (67%) and the women (62%) (χ2 (1, N = 1057) =1.16, p = 0.282).  The women (37%) were more likely than the men (23%) to be noted to have a history of self-harm (χ2 (1, N = 1057) = 11.12, p = 0.001).  Women (38%) also had significantly higher rates of prior victimization experiences than men (20%) (χ2 (1, N = 1057) = 20.25, p < 0.001).     62  Table 4.1  Characteristics of a Forensic Psychiatric Cohort Characteristic Male (n = 929) Female (n= 128) pab  Age, M (SD) 39.76 (12.62) 39.92 (11.94) .887 Legal Status     NCRMD  n (%)      Unfit      Probation      Other      Unknown (missing)  294 (32%) 15 (2%) 242 (26%) 68 (7%) 312 (34%)  35 (27%) 2 (2%) 43 (34%) 15 (12%) 36 (28%)  .324 .965 .071 .083 .083 Hospital Level of Security     Secure*  n (%)      Closed      Open  126 (48%) 105 (40%) 31 (12%)  N/A 18 (60%) 12 (40%)  N/A   Primary Diagnosis        Schizophrenia Spectrum 523 (54.2%) 46 (34.8%) .001      Mood Disorders 132 (13.7%) 33 (25.0%) .001      Substance Use Disorders 89 (9.2%) 16 (12.1%) .292      Childhood/Adolescent Onset 27 (2.8%) 2 (1.5%) .390      Anxiety Disorders 26 (2.7%) 11 (8.3%) .001      Personality Disorder/Traits 14 (1.5%) 9 (6.8%) .001      Cognitive Impairments 13 (1.3%) 1 (0.8%)     .572      Traumatic Brain Injury 8 (0.8%) 0 (0%) .292      Other 69 (7.2%) 7 (5.3%) .433      Not yet diagnosed 64 (6.6%) 7 (5.3%) .555 Note. N = 1057. N/A = not applicable. *Due to the small number of female patients there are no women housed in our secure units. ab t-test or chi-square test.  4.2.3 Setting 4.2.3.1 Implementation of the START in the BC FPSC During the development of the START and the publication of the manual the authors consulted with staff and presented regularly via colloquia and invited lectures at institutional training days.  Initial workshops were conducted with psychiatry, psychology, and case  63  managers to obtain grassroots buy-in and integrate direct-care provider feedback in revising and improving the tool.  In 2006, organiziation-wide training and implementation of the START was initiated at the Forensic Psychiatric Hospital (FPH) and the six Regional Forensic Clinicss spread through-out the province.  Between 2006-2008, all case managers, social workers, nurses, and psychiatrists at the hospital and clinics were trained.   Refresher training was subsequently provided across the entire organization in 2010.  Our multidisciplinary START Implementation Team maintains that these efforts are essential to support the fidelity of the uptake and integration of the measure into practice.  A START workshop is also presented at the local FPSC conference each year, offering yet another opportunity for staff to receive refresher training and for new staff to obtain an introduction to the measure.  START training is now offered as core competency training to all new employees who are direct-care providers and, consistent with the recommendations in the Department of Health document (2007) discussed above, the FPSC policy now requires that all direct-care staff receive a half-day of refresher training at two-year intervals.  The START was first introduced into practice as a means of integrating client strengths, supporting transparent, valid, and reliable assessments of diverse risks that support the development of comprehensive and accountable management and treatment plans.  As the full organization became trained the 'grassroots' efforts to support uptake and integration into practice were further supported by incorporating START into policy and practice.  Specifically, a current START is required when treatment teams make a request to the Program and Privileges committee for patients to have their first community access.  Similarly, our Day Leave decision-making protocol requires that nurses examine the START prior to approving a patient to access the community on day leave from the hospital.    64   4.2.4 Procedures  This study received institutional approval both from the FPSC Research Advisory Committee and the University of BC Research Ethics Board.  In line with FPSC policy, START assessments are completed by the individual’s treatment team within one month of an individual’s admission to the FPH and within three months of admission to the Regional Forensic Clinics.  The original form is placed on the client’s file and a copy of the START summary sheet is provided to the research department where a research assistant enters the data into an SPSS file.  We made a request to the data administrator for a list of all unduplicated START forms administrator available for FPSC.  The FPSC quality staff then provided the research team with a scrubbed (de-indentified) SPSS file containing all of the START form data for the agency from September 1, 2009 to August 31, 2010.   4.2.5 Measures START (Webster et al., 2004, 2009).  START comprises 20 items and contains two additional case specific items.  Each item is intended to be rated simultaneously as both a vulnerability and a strength, with scores ranging from 0 (minimal vulnerability/strength) to 2 (definite vulnerability/strength).  In addition, clinicians are encouraged to complete a careful assessment of historical factors known to be associated with risk of harm to self and others.  After coding each dynamic item and considering the historical factors, the evaluator makes specific risk estimates (i.e., low, moderate, or high) for seven adverse outcomes (violence to others, suicide, self-harm, self-neglect, unauthorized absence, substance use, being victimized).  In clinical practice, the START items are not summed and there are no cut-offs; however, for  65  research purposes the strength and vulnerability scores can be summed.   4.3 Results 4.3.1 START Reliability The START strength scores and the vulnerability scores were found to maintain good structural reliability when implemented into practice in this forensic service.  When coded by multidisciplinary treatment teams, the Mean Inter-item Correlations (MIC) were .46 and .40, for strengths and vulnerabilities, respectively.  The Mean Corrected Item-Total Correlation (MCITC) were .50 (strengths) and .48 (vulnerabilities).  These results indicate a high degree of item homogeneity.  Overall, the internal consistency of the strength and vulnerability scales also was acceptable with Cronbach’s Alpha = 0.80 for strengths and .76 for vulnerabilities.  4.3.2 Descriptive Statistics In our sample of 929 men and 128 women, the results indicated excellent dispersion across the total range of item (0, 1, or 2) and scale scores (0-40) (Tables 4.2 and 4.3).  The dispersion and the mean strength and vulnerability scores in our forensic psychiatric service are nearly identical for male and female patients.  Overall, the mean strength score for the men was 19.28 (SD = 7.72, range = 0-39).  Similarly, the mean strength score for women was 19.96 (SD = 8.79, range = 0-40), out of a possible range of 0-40. Few START item scores (strength or vulnerability) were found to be coded significantly differently for male versus female clients.  That is with the exception of four variables all found to be significantly different in two-tailed t-tests: recreation (p = 0.010), social skills (p = 0.008), and substance use (p = 0.019) on the strengths side and emotional state on the vulnerabilities side (p = 0.003).    66   Table 4.2  START Scales by Gender Patient gender / START Scale M (SD) Range Men (n= 929)            Strength 19.28 (7.72) 0-39      Vulnerability 17.11 (7.90) 0-40 Women (n= 128)            Strength 19.96 (8.79) 2-39      Vulnerability 16.89 (8.89) 0-38 Note. N = 1057. Items are coded as 0, 1, 2. The possible range for START total scores = 0-40. Higher scores indicate more strengths or more risks.  Table 4.3  Frequency of START Item Endorsement Among Forensic Inpatients Item   n Distribution M Frequency of Endorsement (%) (SD) Minimally Present (0) Moderately Present (1) Maximally Present (2) 1. Social Skills  Strength  300 0.95 (0.50) 15% 75% 10% Vulnerability 300 1.01 (0.54) 14% 71% 15% 2. Relationships Strength 301 0.77 (0.48) 26% 72% 3% Vulnerability 300 1.07 (0.57) 13% 67% 20% 3. Occupational Strength 299 0.76 (0.63) 34% 55% 10% Vulnerability 298 1.07 (0.70) 21% 51% 28% 4. Recreational Strength 300 0.88 (0.59) 24% 64% 12% Vulnerability 299 0.89 (0.61) 24% 62% 13% 5. Self-Care  Strength 301 1.25 (0.65) 12% 52% 37% Vulnerability 301 0.63 (0.60) 43% 51% 6% 6. Mental State Strength 301 0.81 (0.60) 30% 60% 10% Vulnerability 300 0.95 (0.69) 27% 52% 21% 7. Emotional State Strength 301 0.96 (0.45) 12% 79% 8% Vulnerability 301 0.91 (0.59) 22% 65% 13% 8. Substance Use Strength 300 1.42 (0.75) 16% 26% 58% Vulnerability 300 0.54 (0.76) 63% 21% 16% 9. Impulse Control Strength 301 0.94 (0.62) 22% 62% 16% Vulnerability 301 0.86 (0.67) 31% 53% 17% 10. External Triggers Strength 298 0.94 (0.59) 21% 64% 15% Vulnerability 298 0.85 (0.68) 32% 51% 17% 11. Social Support Strength 301 0.83 (0.55) 25% 67% 8%  67  Item   n Distribution M Frequency of Endorsement (%) (SD) Minimally Present (0) Moderately Present (1) Maximally Present (2) Vulnerability 300 1.11 (0.56) 11% 68% 21% 12. Material Resources Strength 301 1.02 (0.58) 16% 66% 18% Vulnerability 301 0.84 (0.58) 26% 64% 10% 13. Attitudes  Strength 300 0.84 (0.56) 25% 66% 9% Vulnerability 300 0.94 (0.56) 19% 68% 13% 14. Medication Adherence Strength 299 1.40 (1.33) 11% 54% 32% Vulnerability 298 0.73 (1.43) 53% 38% 7% 15. Rule Adherence Strength 300 1.12 (0.66) 16% 55% 29% Vulnerability 300 0.73 (0.66) 39% 49% 12% 16. Conduct  Strength 299 1.08 (0.64) 17% 59% 25% Vulnerability 300 0.66 (0.64) 43% 48% 9% 17. Insight Strength 300 0.49 (0.56) 54% 43% 3% Vulnerability 299 1.42 (0.62) 7% 44% 49% 18. Plans Strength 301 0.74 (0.57) 33% 61% 7% Vulnerability 301 1.09 (0.61) 14% 62% 24% 19. Coping Strength 301 0.87 (0.52) 21% 72% 7.6% Vulnerability 301 1.02 (0.53) 13% 72% 15% 20. Treatability Strength 297 0.82 (0.50) 23% 72% 5% Vulnerability 300 1.10 (0.49) 7% 75% 17% Note. N = 1057.  4.3.3 START Across the Continuum of Care When we examined the START scale (i.e., total strengths scores and total vulnerabilities scores) and item scores by setting and security level we generally see the scores differ in the manner expected (see Table 4).  That is, patients in the maximum security units (n = 126) have the highest vulnerabilities scores, M = 21.44 (SD = 6.96, range = 4-40, and lowest strength scores, M = 15.79 (SD = 6.18, range = 3-29).  The patients in the medium security units (n = 123) had scores that fell between those of the maximum units and minimum units: for strengths, M =15.73 (SD = 4.73, range = 7-32); and for vulnerabilities, M = 16.97 (SD = 4.67, range = 6-29).  In our minimum secure units (n = 43) the mean strength score was 26.12 (SD = 5.43, range = 15-36) and the mean vulnerability score was 11.20 (SD = 6.16, range = 1-20).  In sum, the  68  differences across the hospital security levels show that the more privileges, freedoms, and community access enjoyed by the patient the lower the vulnerabilities score (F(2, 289) = 50.62, p < 0.001) and the higher the strengths score (F (2, 291289) = 58.60, p < 0.001).  Among patients in the maximum security units, the range of vulnerability scores was 4-40 but that was reduced to a range of 6-29 in the medium security units.  In the minimum security units the highest vulnerability score was half the possible total range on the START (i.e., 20 out of a possible score of 40).  Similar differences were also noted for the strength scores; shifting from a maximum total score of 29 to 32 and finally 36 in the maximum, medium and minimum security wards, respectively (see Tables 4.4).  It is also promising to see that the mean item scores also consistently reflect the anticipated mean differences as patients’ progress through the security levels (Table 4.5).  Although preliminary, the data potentially suggest that the START may be a useful method for monitoring treatment progress and making recommendations for placement and privileges. Table 4.4  START Scale Scores by Setting and Security Level Security Level  M (SD) Range Secure (n= 126)         Strength 15.79 6.18 3-29      Vulnerability 21.44 6.96 4-40 Closed (n = 123)         Strength 19.73 4.73 7-32      Vulnerability 16.97 4.67 6-29 Open (n = 43)         Strength 26.12 5.43 15-36      Vulnerability 11.20 6.16 1-20 Community (n = 763)         Strength 19.54 8.30 0-39      Vulnerability 16.69 8.37 0-38 Note. N = 1057.Setting data were missing from two START forms. Items are coded as 0, 1, 2.  69  The possible range for START total scores = 0-40. Higher scores indicate more strengths or more risks.  Table 4.5  START Item Distributions by Hospital Security Level START Item  START Scale Levels of Security Secure Closed  Open n M  (SD) M (SD) M (SD) F-test p 1. Social Skills 288 Strength 0.80 (0.47) 0.97 (0.46) 1.35 (0.49) 21.82 0.001 288 Vulnerability 1.18 (0.54) 0.91 (0.47) 0.70 (0.50) 17.77 0.001 2. Relationships 289 Strength 0.62 (0.52) 0.84 (0.39) 1.07 (0.40) 17.75 0.001 288 Vulnerability 1.19 (0.60) 1.04 (0.52) 0.72 (0.50) 11.56 0.001 3. Occupational 287 Strength 0.56 (0.61) 0.85 (0.58) 1.14 (0.56) 17.00 0.001 286 Vulnerability 1.26 (0.72) 0.99 (0.64) 0.70 (0.60) 12.39 0.001 4. Recreational 288 Strength 0.67 (0.58) 0.94 (0.55) 1.28 (0.50) 20.47 0.001 287 Vulnerability 1.08 (0.62) 0.81 (0.54) 0.49 (0.51) 18.85 0.001 5. Self-Care  289 Strength 1.17 (0.67) 1.32 (0.63) 1.42 (0.59) 3.13 0.045 289 Vulnerability 0.72 (0.59) 0.58 (0.59) 0.42 (0.59) 4.75 0.009 6. Mental State 289 Strength 0.57 (0.56) 0.97 (0.54) 1.12 (0.63) 22.57 0.001 288 Vulnerability 1.10 (0.72) 0.86 (0.61) 0.60 (0.70) 9.98 0.001 7. Emotional State 289 Strength 0.83 (0.49) 1.00 (0.29) 1.33 (0.48) 23.83 0.001 289 Vulnerability 1.04 (0.63) 0.86 (0.50) 0.56 (0.50) 12.28 0.001 8. Substance Use 288 Strength 1.26 (0.82) 1.47 (0.69) 1.81 (0.45) 9.50 0.001 288 Vulnerability 0.61 (0.83) 0.51 (0.71) 0.33 (0.61) 2.22 0.111 9. Impulse Control 289 Strength 0.76 (0.61) 0.98 (0.55) 1.44 (0.50) 22.87 0.001 289 Vulnerability 1.10 (0.71) 0.74 (0.56) 0.42 (0.50) 22.22 0.001 10. External Triggers 286 Strength 0.80 (0.58) 0.97 (0.59) 1.33 (0.48) 13.84 0.001 286 Vulnerability 0.96 (0.75) 0.84 (0.61) 0.48 (0.55) 8.35 0.001 11. Social Support 289 Strength 0.77 (0.54) 0.84 (0.53) 1.07 (0.55) 4.98 0.007  70  START Item  START Scale Levels of Security Secure Closed  Open n M  (SD) M (SD) M (SD) F-test p 288 Vulnerability 1.23 (0.55) 1.03 (0.53) 0.86 (0.52) 8.88 0.001 12. Material Resources 289 Strength 0.97 (0.63) 0.98 (0.51) 1.37 (0.49) 9.08 0.001 289 Vulnerability 0.94 (0.63) 0.81 (0.53) 0.58 (0.55) 6.16 0.002 13. Attitudes  288 Strength 0.69 (0.57) 0.87 (0.50) 1.26 (0.49) 18.36 0.001 288 Vulnerability 1.13 (0.54) 0.89 (0.49) 0.50 (0.55) 23.59 0.001 14. Medication Adherence  281 Strength 1.05 (0.66) 1.73 (1.75) 1.47 (0.59) 11.19 0.001 280 Vulnerability 0.65 (0.70) 0.46 (0.55) 0.33 (0.57) 5.25 0.006 15. Rule Adherence 288 Strength 0.94 (0.63) 1.15 (0.54) 1.65 (0.57) 21.76 0.001 288 Vulnerability 0.98 (0.67) 0.65 (0.59) 0.19 (0.45) 28.75 0.001 16. Conduct  287 Strength 0.85 (0.63) 1.15 (0.54) 1.67 (0.47) 33.87 0.001 288 Vulnerability 0.92 (0.68) 0.54 (0.52) 0.12 (0.32) 35.13 0.001 17. Insight 288 Strength 0.31 (0.53) 0.58 (0.57) 0.79 (0.47) 15.05 0.001 287 Vulnerability 1.62 (0.56) 1.33 (0.60) 1.05 (0.65) 17.52 0.001 18. Plans 289 Strength 0.60 (0.58) 0.77 (0.56) 1.12 (0.39) 14.79 0.001 289 Vulnerability 1.18 (0.64) 1.10 (0.59) 0.74 (0.44) 8.81 0.001 19. Coping 289 Strength 0.70 (0.56) 0.94 (0.35) 1.28 (0.50) 26.00 0.001 289 Vulnerability 1.19 (0.58) 0.92 (0.33) 0.67 (0.52) 21.75 0.001 20. Treatability 285 Strength 0.70 (0.52) 0.83 (0.44) 1.14 (0.47) 13.16 0.001 288 Vulnerability 1.21 (0.48) 1.07 (0.44) 0.79 (0.47) 13.78 0.001 Note:  Secure, n = 124-126; Closed, n = 115-123; Open n = 42-43  4.3.4 Risk Estimates In formulating opinions regarding the START risk estimates, clinicians are asked to rate the patient as low, moderate, or high risk over the next several months (up to three).  The assessors are instructed to code the START form in a manner that would reflect the client’s level  71  of risk if her or she were released to the community without conditions and “left to their own devices”.  Overall, the majority of the risk estimates were rated as low and there were few gender differences.  For violence risk 7% of men and 5% of women were rated high risk.  Most patients fell into the low risk group (63% of men; 69% of women).  Very few patients were considered high risk for suicide (2% of men and 1% of women) and the vast majority of patients were considered to be low risk (92% of men and 93% of women).  The threat of self-harm was deemed to be only slightly higher with 1% of men and 3% of women rated high risk (low risk = 90% men; 83% women).  The risk of unauthorized leave also was generally thought to be relatively low, with 6% of men and 0% of women rated high risk and 81% of men and 90% of women rated low risk.  Overall, the chi-square results indicated few gender differences.  The substance abuse risk estimates stand out for two primary reasons.  First, it was one of two risk estimates for which there was a significant gender difference, with men being considered a significantly higher risk than women (p = 0.007).  Secondly, the substance abuse risk estimate had the largest proportion of participants in the high risk category (19% of men; 20% of women).  In comparison, 62% of the women and 48% of the men were rated low risk for substance abuse.  Self-neglect was considered low risk for 72% of men and 76% of women.  Seven percent of men and six percent of women were rated high risk for self neglect in the short-term (weeks to months). The only other risk estimate analysis to indicate a significant gender difference was the risk of the patient/client being victimized.  A total of 4% of men and 10% of women were determined to present as high risk for being abused/assaulted or otherwise victimized by others.  Slightly over 78% of the men and 71% of the women were considered to present little risk of being harmed by others.  Finally, the women were significantly more likely be considered at risk of victimization than the men (p = 0.036).   72   4.3.4.1 Co-Occurring Risks A somewhat unique aspect of START, compared to the pre-existing SPJ measures, is the intention for uptake among interdisciplinary direct care providers who it is hoped will implement these structured assessments into daily decision-making.  Moreover, the goal is for those assessments to reflect a consideration of mental health and risk for seven adverse events commonly seen in individuals living with mental illness and/or among institutionalized and marginalized populations (see Webster et al., 2006, 2009).  Pearson correlations were used to examine how much overlap there is in the START risk estimates according to experienced forensic psychiatric treatment teams.  A consideration of the full sample indicated that there is significant overlap across the seven START risk estimates.  For example, the treatment teams’ estimates of the risk of violence was significantly correlated with their determinations of the risk of self-harm (r = .27, p < .001), risk of suicide (r =.27, p < .001), risk of unauthorized leave (r =.47, p < .001), risk of substance abuse (r =.29, p < .001), risk of self-neglect (r =.33, p < .001), as well as the risk of the client being victimized by others (r =.31, p < .001).  We also examined the intercorrelations on the risk estimates for the men and women, separately (see Table 4.6).  The results were quite similar with significant and generally moderate correlations.  A few exceptions are worth specific mention.  There appeared to be substantial overlap between the treatment teams’ concern that women would engage in substance abuse and their concern that she might be at risk for victimization (r =.64, p < .001) when compared to the same result for the men (r =.25, p < .001).  The risk of unauthorized leave and the risk of self-harm were significantly correlated for the men (r =.25, p < .001) but not for the women (r =.10, p = .365).   73  Table 4.6  Intercorrelations of START Risk Estimates as a Function of Client Gender  Correlation START Risk Estimate 1 2 3 4 5 6 7 1. Violence - .32*** .26* .33** .38*** .24* .33*** 2. Self-Harm .27*** - .72*** .10 .36*** .45*** .40*** 3. Suicide .25*** .68*** - .06 .28** .44*** .40*** 4.Unauthiorized Leave .48*** .25*** .18*** - .42*** .24* .19 5. Substance Abuse .27*** .20*** .10** .29*** - .52*** .64*** 6. Self Neglect .34*** .36*** .22*** .50*** .31*** - .68*** 7. Victimization .31*** .09* -.02 .46*** .25*** .47*** - Note. Correlations for female participants (n= 79-102) are presented above the diagonal, and correlations for male participants (n= 582-795) are presented below the diagonal.  *p≤.05, **p≤ .01, ***p ≤ .001.  4.3.4.2 Key Items and Critical Items Consistent with previously well-established SPJ measures such as the HCR-20 (Webster et al., 1997), the START includes the option for assessors to indicate if an item is considered to present a particular strength (a ‘key item’) or a particular vulnerability (a ‘critical item’).  A key item is a potential therapeutic lever, a characteristic of the client that is believed to represent a particular asset that might facilitate recovery and stability.  Key items can include evident capacities (i.e., characteristics or assets already present in the individual’s character or their surroundings) as well as emerging strengths that the client’s team would want to focus on in the treatment plan.  In contrast, a critical item indicates a ‘red flag’, a variable that is known through past evidence (or is believed to have the potential) to seriously derail the patient’s progress or  74  increase their risk for an adverse event or mental health deterioration (see Webster et al., 2006, 2009).   Table 4.7 presents the frequency with which the 20 START items are endorsed as key items or critical items.  A perusal of the results reveals a few primary findings.  First, consistent with what we have seen at the item and scale level, there is considerable dispersion, each of the items are coded as both key items and critical items.  Second, as would be expected, there are a handful of items that are more commonly considered to be critical items.  Specifically, mental state (38.4%), emotional state (21.6%), substance use (44.2%), impulse control (21.9%), medication adherence (17.9%), and insight (26.8%) are common critical items.  In comparison, there appears to be less of a preference for specific key items; though as might be expected, social support is the most commonly coded key item (14.7%).  Third, although there is a clear indication that some items are more likely be coded as critical items (e.g., substance use, insight) than as key items, there are no items for which we do not see representation on both sides of the scale.    Table 4.7  Frequency of START Items being Endorsed as Key Items or Critical Items Key Item (%) START Items Critical Item (%) 7.4% 1. Social Skills 9.3% 8.6% 2. Relationships 9.7% 11.8% 3. Occupational 7.6% 4.1% 4. Recreational 2.9% 6.9% 5. Self-Care 4.7% 6.2% 6.  Mental State 38.4% 3.8% 7. Emotional State 21.6% 12.8% 8. Substance Use 44.2% 2.9% 9. Impulse Control 21.9% 2.2% 10. External Triggers 14.2% 14.7% 11. Social Support 8.9%  75  Key Item (%) START Items Critical Item (%) 5.0% 12. Material Resources 5.2% 3.3% 13. Attitudes 8.4% 9.8% 14. Medication Adherence 17.9% 3.4% 15. Rule Adherence 5.9% 1.9% 16. Conduct 6.3% 5.3% 17. Insight 26.8% 2.6% 18. Planning 6.3% 2.2% 19. Coping 9.2% 3.1% 20. Treatability 5.9% Note. N = 1057  4.3.4.3 Signature Risk Signs The concept of signature risk signs evolved from the serial sexual homicide literature and we must also credit our colleague Dr. Emlene Murphy, former medical director for raising the issue with the START team.  In the same way that a detective coming upon a serial sexual homicide scene might recognize tell-tale signs that point to clues left consistently by a particular offender (the placing of the body in a certain position, the removal of a trophy, leaving a playing card or a flower at the crime scene) in the mental health field, experienced clinicians can often point to seemingly benign or unrelated behaviors that signal impending mental health deterioration or an escalation in risk (see Webster et al., 2009).  Noteworthy, the START authors have always asserted that these were expected to be rare.  In fact, however, the data revealed that in 1 of 3 cases (27%) a signature risk sign was reported by the clinical team in our service.  Examples of these included an individual who begins to wear a traditional burka, another client who yells “Tink!” and another who becomes preoccupied with Diana Krall.  The objective of documenting signature risk signs is to ensure that important information about a client’s cycle of offending or their mental health cycle is not overlooked.  This is also relevant of course to the  76  risk formulation section at the bottom of the START form.    4.3.4.4 THREAT Boxes (Threat of Harm that is Real Enactable Acute and Targeted Also relatively unique to this measure, the START form includes direct and explicit reference to the type of scenarios that reflect the need to implement ‘Tarasoff warnings’ (Tarasoff v. Regents of the University of California, 1976; see Borum & Reddy, 2001) and to take immediate action to ensure the safety of the client or others (see Webster et al., 2006, 2009 for a discussion of THREAT boxes).  Much like the signature risk signs, we anticipated that an indication of acute and targeted THREATs would be relatively rare.  The data suggest that although the TREAT boxes are not commonly coded as being present, they are not particularly rare either.  There were no significant gender differences noted when we ran chi-square comparisons.  A THREAT of violence was noted in 7.4% (n = 70) of cases with male clients and 6.0% (n = 7) of cases with female clients (p = .367).  Self-harm was noted to be imminent and requiring immediate intervention in 3.1% of men (n= 24) and 5.4% of women (n = 6) (p = .367).  Finally, an imminent suicide THREAT requiring immediate response was reported by 2.6% of clinical teams working with male clients and 2.8% of clinical teams completing START for female clients.   4.4 Discussion 4.4.1 What is the Clinical and Risk Profile of a Large Canadian Forensic Psychiatric Cohort? Reflecting the value of dynamic measures to inform risk assessment and management, the vast majority of our client population presents with a history of many of the risk factors that  77  would commonly be used to distinguish a group of individuals at high risk for negative events.  As is likely the case in most forensic systems, the vast majority of our patients are young males, many of whom have a documented history of long-term mental health problems, substance abuse, poor occupational and educational attainment, and specific incidents of the risk estimates in question (e.g., 83% of the men and 77% of the women have a history of violence) (also see Nicholls et al., 2009).  For these reasons, it is particularly promising to see considerable dispersion on the START items, suggesting that a snapshot of the client’s current presentation on the START might be a useful means of distinguishing client’s risk levels and informing appropriate risk management and treatment planning efforts in the short-term.  Consistent with the design of the START, the treatment teams’ risk estimates evidenced considerable intercorrelations, suggesting that there is substantial overlap in the adverse events commonly a focus of clinical care in the forensic setting.    4.4.1.1 Profile as a Function of Gender Our results suggest that the clinical characteristics and risk-needs profile of Canadian male and female forensic psychiatric patients present with many similarities; that being said, some important gender differences are evident, as well.  Men were reported to have significantly higher rates of schizophrenia spectrum disorders and women had higher rates of anxiety disorders, mood disorders, and personality disorders.  As would be expected, we found very high rates of substance abuse histories among both male (67%) and female (62%) patients.  The women (37%) were more likely than the men (23%) to have a history of self-harm and prior victimization experiences than men (38%, 20%, respectively).  In the present study, START item scores (strength and vulnerability) were found to result in similar mean scores across the  78  genders.  With few exceptions we found the mean scale scores (i.e., total vulnerabilities and total strengths) and risk estimates were quite similar across the genders, as well.  These findings are somewhat unique from what has been reported in prior research with forensic samples of men and women using other measures relevant to risk assessments (Douglas, Strand, Belfrage, Fransson, & Levander, 2005) and what we have found in our own research (Nicholls, 2001) in this setting with the HCR-20 (Webster et al., 1997) and PCL:SV (Hart et al., 1995).  The results are, however, consistent with our prior START research in assessments conducted by research assistants in civil psychiatric patients (Petersen, Douglas, & Nicholls, 2011).   4.4.1.2 Profile as a Function of Setting and Security Level The range of scores and the mean scores across the care pathway in the hospital reflects what we would expect to see with START scores.  At both the item and scale level START strength scores increase and vulnerabilities decrease, the less secure the unit.  As we would expect, the results for the comparisons across the hospital units offer a gross indication of convergent validity.  We caution readers to view this finding within the context of the setting where we completed the study.  Specifically, given that START is now being used clinically in the hospital one would expect these matters to be strongly correlated given it is the treatment team members who complete the START and also who make recommendations for placement.  However, we would assert the data does still offer some evidence of convergent validity.   In comparison to the findings across the inpatient units, when we additionally examined the START scores in the community clinics the results were not what we expected.  We had initially anticipated that this would reflect just one further step in the clinical pathway (i.e., moving from open units which offer community access through the security levels at the hospital to  79  independent living); in fact, the results suggest this is not the case.  Specifically, the range of scores was much wider and the vulnerabilities scores were higher and the strengths scores were lower than we would expect to see if this was just one more step in the clinical pathway.  The results may largely reflect the nature of the community population in our service.  Many of the FPSC community clients have never been inpatients at the hospital and in fact it is a minority of those clients who are former NCRMD patients discharged from the hospital.  As we noted previously many of the clinic clients are probationers.  The results point to further areas of inquiry including running these analyses for former FPH patients under community clinic supervision versus clinic clients who have never been at FPH.  The findings also lead us to query whether perhaps the community clinic treatment teams are operating with a different threshold because a ‘wrong decision’ might have more immediate and disastrous consequences? It is also the case, of course, that the community clients are exposed to more destabilizers and care providers might see a wider range of risks than we expected.  In order to understand what these unexpected findings mean we are currently comparing and contrasting START assessments completed by content experts (research assistants with considerable training and supervision) with START assessments completed on the same clients by their treatment team.  This examination of field reliability will shed light on the extent to which START has been implemented with fidelity and offer insights into whether clinic patients truly do present with higher risk than we would have predicted (Viljoen, Launeanu, Hendry, Nicholls, & Brink, 2011).  We will additionally disaggregate the community clients into former FPH patients and those who went directly to supervision with the clinics.    80  4.4.2 What are the Psychometric Properties of the START in a Forensic Service when the Evaluations are Completed by Clinical Teams? To summarize the descriptive findings, in both analyses as a function of the sex of the participant and as a function of setting/security level we found considerable dispersion of scores at the item, scale, and risk estimate level.  This suggests that the START may prove useful for informing determinations about care setting and treatment needs in forensic populations.  Prior research with the START suggests structural reliability appears to be strong (see Webster et al., 2009).  This study relied on START assessments by actual clinical teams while our prior START publications (Desmarais et al., 2011; Wilson et al., 2010, 2011) have generally relied on researcher-informed START assessments.  Results indicating strong structural item homogeneity and internal consistency in this field study are considered to be cautiously promising.  It is difficult to determine what might be the optimal structure of a measure such as the START given that it is intended to address mental health and client safety outcomes which might reflect a common underlying fragility and dysfunction among individuals who manifest the range of behaviors covered-off by the START (risk to self and others broadly, as well as risk of being victimized).  Alternatively, it might be argued that given the diversity of the outcomes of interest we might hope to see less homogeneity across the items than typically seen on other SPJ measures that reflect an assessment for a single outcome, for instance. Evidence that the distribution of item scores, strength/vulnerability scale total scores, and risk estimates are similar to what we have seen in our prior work provides further preliminary evidence of the fidelity of the implementation.  We can begin to have some confidence that the implementation has been at least minimally successful.  This is, however, very preliminary evidence and rigorous research evaluating the field reliability and fidelity requires us to complete  81  START assessments by our own content experts.  As noted above, we currently have a community clinic study (Viljoen et al., 2011) and a hospital based study (Nicholls, Petersen, & Brink, 2011) in which research assistants with strong interrater reliability are conducting shadow START evaluations to test this further.  We will also be collecting follow-up information that will also allow us to compare and contrast the validity of the two assessments.  4.4.3 Limitations Due to our methodology, we are unable to comment on the participants’ characteristics or risk factors that are not explicitly delineated on the START.  For instance, we were not able to report on prior hospitalizations/mental health history, the nature of the index offence, or prior contact with the criminal justice and forensic systems.  Our reliance on secondary data also prevented us from thoroughly testing the field reliability of the measure, a task we are undertaking currently.  Additionally, we focused on gender and group differences across security levels in the present study.  It will be important in future research to continue to build on the work of Wilson et al., (2011), for instance, and examine within individual change scores.  Finally, we did not examine the validity of START (predictive, incremental, convergent, divergent) and therefore, we were not able to report on important aspects of the psychometric properties of the measure, our current research will allow us to address each of these essential areas of inquiry in the near future.   4.4.4 Conclusion A handful of recent meta-analytic studies suggest that structured violence risk measures, regardless of whether their development reflected a mechanical, algorithmic approach (i.e.,  82  actuarial) or took a conceptual and empirically-informed approach (i.e., structured professional judgment) they are essentially “interchangeable” in terms of their predictive utility (Yang, Wong, & Coid, 2010, p. 759; see also Campbell et al., 2009; Guy, 2008; Singh, Grann, & Fazel, 2011).  Consequently, mental health professionals should turn to a consideration of the measure that best addresses the task put to them and that has been validated in the population(s) for whom they are responsible.  The results of the present study contribute to the developing body of evidence that suggests the START could provide insights into the risk management and treatment needs of forensic patients.  Ethical and professional practice guidelines mandate that well-considered risk assessments include attention to employing methods that help to ensure the highest degree of validity, (accuracy), reliability (consistency), transparency (well-reasoned and clearly articulated rationale for the approach and conclusions) and comprehensiveness.  They should support the need for efficient and clear communication that bridges gaps between care-givers and settings, for instance.  Ultimately, they also should serve to enhance accountability by providing a convenient and concise way to demonstrate clinical progress.  Finally, risk assessments need to be defensible, meaning they should be able to withstand scrutiny.  Acknowledging the inherent unpredictability of future behavior, it is generally accepted that ‘defensibility’ rather than ‘certainty’ is the ultimate objective of a risk assessor.  In much the same manner that Dr. Donald Winnicott, the celebrated pediatrician and psychoanalyst attempted to reduce the anxiety of mothers in the 1950s by assuring them that they need only needed be “good enough” mothers not “perfect” mothers, the same principles might apply to risk assessors.  We are not endorsing mediocrity in risk assessment and risk management but rather acknowledging that there is no way of promulgating a ‘perfect’ risk assessment.  This logic also has been endorsed by the Royal  83  College of Psychologists (2008).  In their ‘Rethinking risk to others in mental health services’ report, the scoping group concluded that although risk management is a core function of all medical practitioners and in some circumstances negative events, including violence, can be prevented or the implications (frequency, severity of outcomes) reduced risk cannot be eliminated.  “Accurate prediction is never possible for individual patients” (p. 9).  Acknowledging the inherent limitations in the risk assessment and management field, mental health professionals should rest easy if they produce well-reasoned and defensible evaluations that cover off the ethical and professional guidelines described above, remain well-versed in the literature and legal context in which they practice, and consult with colleagues when the circumstances require it.  When we developed the START we viewed the field as suffering from four primary limitations: (1) a concentration on static and historical variables to the neglect of dynamic items and treatment relevant variables; (2) a failure to integrate protective factors into assessments and thus treatment planning and management decisions; (3) a lack of attention on the need to attend to diverse patient safety needs commonly seen within marginalized populations (i.e., beyond violence), such as forensic psychiatric patients and (4) forecasting years into the future and failing to integrate the advances achieved to date in the risk assessment field into the work of direct-care providers who are faced with the task of making decisions about patients immediate and short-term needs.  Although advances in research have been taken up with enthusiasm often, the most accurate description of clinical decision making at a unit level, on a day–to-day basis would be unstructured clinical judgment.  Our experience has been that the START achieves our objective of addressing some of these limitations.  In particular, we hope it contributes to bridging the gap between what we know about risk assessments and direct-care provider clinical  84  responsibilities in the short-term (e.g., day leaves, security levels, and privileges).  The primary challenge that remains, however, is to examine the capacity of START to inform the reduction of risk.   85  Chapter 5:   The Psychometric Properties of START:  Comparing and Contrasting a Forensic and a Civil Psychiatric Inpatient Cohort  5.1 Introduction’ The Short-Term Assessment of Risk and Treatability (START; Webster, Martin, Brink, Nicholls, & Desmarais, 2004, 2009) is a novel risk assessment measure which considers both clients’ strengths and vulnerabilities in order to estimate the risk for seven interconnected adverse outcomes: violence, suicide, self-harm, self-neglect, unauthorized absence, substance use, and victimization (Webster et al., 2006).  There is a growing body of literature examining START in forensic settings, demonstrating the clinical and administrative utility of the measure.  There is also considerable evidence of the utility of START assessments as valid and reliable predictors of violence.  However, as a recent systematic review concluded (O’Shea & Dickens, 2014) there have been only a handful of studies which have considered the psychometric properties of START in a civil psychiatric sample or considered outcomes other than violence (see Table 1). In addition, the existing research considering civil samples or outcomes other than violence, has been limited to small sample sizes and/or has not considered the utility of risk estimates, the most fundamental part of a structured professional judgement (SPJ) measure.    Table 5.1  Summary of Relevant START Literature Study N Setting START Coding Context Outcomes Considered Abidin et al., 2013 98 Secure forensic hospital Coded for research Harm to others Self-harm Gray et al., 2011 44 3 inpatient units – 1 forensic, 2 civil psychiatric Coded for research Harm to others Self-harm Self-neglect Victimization  86  Study N Setting START Coding Context Outcomes Considered Braithwaite et al., 2010 34 Civil psychiatric hospital Coded  for clinical use Aggression toward others Self-harm Suicide Unauthorized leave Substance use Self-neglect Victimization  Gray et al. (2011) (N = 44) included both civil and forensic inpatients in their research and found that START summary risk judgements were a good predictor of all outcome behaviours they assessed (violence to others, self-harm, self-neglect, and being victimized).  By comparison, strength and vulnerability total scores were only predictive of violence, providing support for the value of the structured professional judgment model.  Abidin et al. (2013) sampled forensic inpatients and found START strength and vulnerability scores to be predictive of violence, however, neither were associated with self-harm.  Unfortunately, Abidin et al. did not use the START as intended, failing to examine the association between the summary risk estimates and the outcomes.  Braithwaite et al. (2010) (N = 34) considered the psychometric properties of START in a civil psychiatric hospital and found START strength, vulnerability and risk estimates to be variably predictive of aggression toward others, self-harm, suicidality, unauthorized leave, substance use, self-neglect and victimization.  A recent systematic review and meta-analysis identified the need for additional research encompassing the complete range of outcomes assessed by START and noted the limitations in the diversity of populations used in the validation research on the measure to date (O’Shea & Dickens, 2014).  In addition, previous research has demonstrated that it is hard to statistically model START strength and vulnerability scores together (e.g., Viljoen et al., 2011; Wilson et al., 2010;) due the high correlations between  87  these two dimensions that leads to multicolinearity.  An exploration of an alternative approach, cluster analysis, is considered.  In addition to suggesting a potentially useful statistical approach, a cluster analytic approach could prove helpful clinically by providing clinicians with information on how strength and vulnerability scores function together holistically which may aid in decision making.  Additionally, from a research prospective, producing a unified model including both strength and vulnerability scores that could be tested would further our understanding of the validity of START. 5.2 Method This research was approved by the research ethics or research review committees of the relevant universities and health agencies. 5.2.1 Sample The current study compared the psychometric properties of START in a forensic and a civil psychiatric sample (N = 208).  All patients at the tertiary civil psychiatric hospital identified for transfer to smaller tertiary psychiatric facilities as part of the redevelopment of psychiatric services in BC were approached for participation.  All participants provided written informed consent prior to participation.  Patients deemed incapable of providing informed consent by their psychiatrists were not included in this study (N = 68).  Nearly all patients approached agreed to participate (N = 106, 91.67%).  This number does not include six patients who left the hospital before we were able to invite them to participate.  The data collection at the forensic psychiatric hospital site was entirely file based and did not require written consent.  All patients who had an up-to-date START on file, as per hospital policy, between January 1, 2009 and December 31, 2011 were included (N = 102).  88  5.2.2 Setting The forensic cohort resided in British Columbia’s (BC) only forensic psychiatric hospital (FPH) which serves the entire province.  Individuals are treated or assessed at FPH if they are found Not Criminally Responsible on account of Mental Disorder (NCRMD); if they are referred for assessment or treatment from jail or prison; or if they are found unfit to stand trial.  FPH provides care to patients on secure, closed, and open units within one campus.  The participants in this study were from all security levels (45% secure, 41% closed, and 14% open).  At baseline, the civil psychiatric cohort resided at a large tertiary psychiatric hospital, Riverview Hospital (RVH).  RVH was set on a large 1,000 acre campus surrounded by forested grounds and provided a range of care from short-term intense treatment to longer-term recovery oriented treatment.  Follow-up data was collected at several smaller community-based tertiary facilities, where patients moved as part of the redevelopment of tertiary psychiatric services across BC (also see Lesage et al., 2008; Petersen et al., 2013).  Patients were transferred to facilities offering the same level (or higher levels) of care than they had been receiving at RVH, for example, patients being treated in a locked ward at RVH were transferred to a locked community facility. 5.2.3 Procedures At baseline, research assistants with a minimum of a BA in a relevant discipline collected demographic, clinical and contextual information from files and used this information to complete a START assessment.  All research staff were trained by an author of START.  Six months of follow-up data was collected by a researcher blind to the baseline START data.    89  5.2.4 Measures A brief, purpose-built demographic package was utilized to collect basic participant information.   Short-Term Assessment of Risk and Treatability (START; Webster, Martin, Brink, Nicholls, & Desmarais, 2004, 2009).  START is a concise clinical guide for the dynamic assessment of seven adverse patient safety outcomes (violence, suicide, self-harm, self-neglect, unauthorized absence, substance use, and victimization).  START guides clinicians toward an integrated, balanced evaluation of the patient’s risk over the short-term (weeks to months).  Twenty items are scored concurrently as both vulnerabilities and strengths, from 0 (not relevant) to 2 (definitely relevant).  Each of the seven risk domains are subsequently assessed as low, moderate, or high risk.  Total scores for both strengths and vulnerabilities can be calculated for research purposes.  START has been found to have good internal consistency (alpha > 0.80), appropriate item homogeneity (MIC = 0.20 - 0.50), and good to excellent inter-rater reliability (intraclass correlation coefficient  > 0.80) (Nicholls et al., 2006; Wilson et al., 2010).  START Outcome Scale (SOS; Nicholls, Gagnon, Crocker, Brink, Desmarais, & Webster, 2007).  In 2007, the Overt Aggression Scale- Modified (OAS-M; Yudofsky et al., 1986) was further modified to capture the START risk domains that were not included on the OAS-M (such as unauthorized leave, substance use, victimization, etc.).  This modified measure, referred to as the START Outcome Scale (SOS, Nicholls, et al., 2007), is intended as an outcome measure for clinical practice and START validation research.  The SOS is an instrument used for assessing the occurrence of diverse challenging behaviours commonly found amongst seriously mentally ill individuals, and captures 9 outcomes: 1) verbal aggression, 2) physical aggression against property, 3) physical aggression against self, 4) physical aggression against other people, 5) self- 90  neglect, 6) substance abuse, 7) victimization, 8) unauthorized leave (UAL), and 9) suicidal behaviour.  Each category is rated on a 4-point scale from least (1) to most (4) severe.  Prior research shows that the START Outcome Scale can be reliably coded from patient files (ICC = .70; Nicholls et al., 2006).  5.2.5 Analysis Data analysis was conducted using SPSS (version 16.0).  Descriptive comparisons were computed with independent samples t-tests, chi-square tests, and odds-ratios.  Inter-rater reliability was computed on approximately 15% of files in both samples.  Cohen’s kappa was utilized for categorical variables (i.e., the SOS) and Interclass correlation (ICC) (two-way random effects, absolute agreement computation for single measures) was computed for continuous variables (i.e., START strength and vulnerability total scores).  Receiver Operator Characteristic (ROC) analyses were used to examine the predictive accuracy of START assessments six months post assessment.  ROC analysis was chosen because they are less influenced by low base rates than other indices such as positive predictive power, negative predictive power, sensitivity, and specificity (Mossman, 1994).  In addition, ROC analysis has become the one the most frequently used measures of accuracy among violence risk researchers (Jackson& Guyton, 2008).  Cluster analysis was conducted as an exploratory method to identify latent homogenous groups of patients using START strength and vulnerabilities scores.  Cluster analysis was conducted using SPSS’s two-step cluster analysis with log likelihood as the measure of distance and Schwarz’s Bayesian Criterion (BIC) as the clustering criterion (SPSS Inc., 2001).  Two step cluster analysis was chosen for its ability to incorporate both continuous and categorical variables.  Cluster models were run allowing SPSS to determine the number of clusters as well as specifying the number of clusters in order to ascertain which model was most  91  clinically relevant and statically robust.  The file was split for the cluster analysis in order to investigate whether the two samples would produce similar or different cluster structures.   5.3 Results 5.3.1 Sample The vast majority of the forensic sample was male, whereas the civil psychiatric sample was more gender balanced.  Both groups were primarily Caucasian, in their 40s, and had received a diagnosis in the schizophrenia spectrum (see Table 1; for more details see Petersen et al., under review). Table 5.2  Socio/Demographic Characteristics at Baseline Characteristic Civil Psychiatric Inpatients Forensic Inpatients Gender      Male      Female  65 (61.3%) 41 (38.7%)  93 (91.2%) 9 (8.8%) Age 47.04 40.47 Ethnicity      Caucasian      First Nations      Asian      Other  80 (75.5%) 8 (7.5%) 9 (8.5%) 9 (8.5%)  75 (73.5%) 5 (4.9%) 8 (7.9%) 14 (13.7%) Legal Status      NCRMD      Remand      Unfit      Involuntary      Voluntary  0 (0.0%) 0 (0.0% 0 (0.0%) 100 (95.2%) 5 (4.8%)  92 (90.2%) 2 (2.0%) 3 (2.9%) 5 (4.9%) 0 (0.0%) Primary Diagnosis      Schizophrenia Spectrum      Other  94 (88.7%) 12 (11.3%)  82 (80.4%) 20 (19.6%) Dual Diagnosis      Substance Use      Personality Disorder      Cognitive Impairment  35 (33.0%) 33 (31.1%) 36 (34.0%)  62 (61.0%) 41 (40.2%) 25 (24.5%)  5.3.2 Inter-Rater Reliability The ICC for START strength total scores within the forensic sample was 0.63, and for the civil sample it was 0.71.  The ICC for START vulnerability total scores for the forensic inpatients was 0.85, and for civil inpatients was 0.60.  According to the critical values for  92  interpreting ICCs (single measure) reported by Fleiss, (1981; ICC <.39 = poor, .40 to .59 = fair, .50 to .74 = good, and ICC > .75 = excellent) these ICCs are good to excellent.  The Cohen’s Kappa Coefficient for the SOS outcomes assessed ranged from 0.60 (verbal aggression) to 1.00 (self-harm, self-neglect, UAL) for the forensic inpatients and 0.33 (property aggression) to 1.00 (self-harm) for the civil inpatients.  According to the benchmarks set by Landis and Koch (1977, 0 is considered”poor,” and coefficients ranging from:  slight - 0.00 to 0.20; “fair” - 0.21 to 0.40; “moderate” - 0.41 to 0.60; “substantial” - 0.61 to 0.80; and “almost perfect” - 0.81 to 1.00); thus, the Kappas for forensic patients ranged from moderate to almost perfect, and fair to almost perfect for the civil sample. 5.3.3 Base-Rates of Risk Outcomes The base-rates of the negative outcomes in the six months following the START assessment are presented in Table 3.  For both samples, the base-rates of all outcomes, with the exception of self-harm and suicidal behaviour are relatively high.  The civil psychiatric inpatients experienced a greater prevalence of most outcomes including: verbal aggression, aggression against others, inappropriate sexual behaviour, self-neglect, unauthorized leave and victimization (for more details regarding and the severity of outcomes see Petersen et al., under review). Table 5.3  Base-Rates of Outcomes Outcome Civil Psychiatric Inpatients (N/% Yes) Forensic Inpatients (N/% Yes) p-Value (chi-square/Fisher’s) Verbal Aggression 82 (77.4%) 56 (54.9%) <0.001 (11.74) Aggression Against Property 43 (40.6%) 30 (29.4%) 0.09 (2.84) Physical Aggression Against Others 54 (50.9%) 36 (35.3%) 0.02 (5.19) Self-Harm  13 (12.3%) 14 (13.7%) 0.75 (0.10) Suicide Behaviours 7 (6.6%) 4 (3.9%) 0.54 (0.75) Self-Neglect  86 (81.1%) 64 (62.7%) 0.003 (8.74)  93  Outcome Civil Psychiatric Inpatients (N/% Yes) Forensic Inpatients (N/% Yes) p-Value (chi-square/Fisher’s) Unauthorized Leave  60 (56.6%) 14 (13.7%) <0.001 (41.70) Substance Abuse 26 (24.5%) 25 (24.5%) 1.00 (0.00) Victimization  60 (56.6%) 36 (35.3%) 0.002 (9.50) 5.3.4 Relationship between START Total Scores and the Adverse Outcomes The mean and range of strength and vulnerability total scores on the START were comparable for the two groups (Strength Scores - civil:  M = 13.27, SD = 6.49; forensic:  M = 13.37, SD = 7.59; Vulnerability Scores – civil: M = 19.30, SD = 6.35; forensic:  M = 17.63, SD = 7.94). The mean strength score was significantly lower for civil psychiatric inpatients who had engaged in the following behaviours: physical aggression, suicidal behaviour, and self-neglect than for patients who had not engaged in those behaviours. By comparison, the mean strength score was significantly lower for forensic psychiatric inpatients who engaged in all outcome behaviours except self-harm, substance abuse, unauthorized leave, and victimization. The mean vulnerability score for civil inpatients was only significantly higher for patients who had engaged in verbal aggression and self-neglect.  In contrast, the mean vulnerability score was significantly higher for forensic patients who had engaged in several adverse behaviours, with the exception of self-neglect, unauthorized leave, substance abuse, and victimization (see Table 4).   Table 5.4  Mean Strength and Vulnerability Total Scores of Inpatients Who Experienced Adverse Outcomes Outcome Civil Psychiatric Inpatients (M/SD)) Forensic Inpatients (M/SD) Verbal Aggression      Strength Score      Vulnerability Score  13.07 (6.72) 19.98 (6.03)*  10.32 (6.16)*** 20.75 (7.18)*** Aggression Against Property      Strength Score      Vulnerability Score  11.81 (6.71) 20.59 (6.90)  7.17 (5.40)*** 22.70 (6.35)*** Any Aggression Against Others      Strength Score   11.46 (6.14)***   9.70 (6.25)***  94  Outcome Civil Psychiatric Inpatients (M/SD)) Forensic Inpatients (M/SD)      Vulnerability Score 20.26 (6.77) 20.76 (7.65)*** Any Self-Harm       Strength Score      Vulnerability Score  13.46 (6.37) 16.92 (4.82)  10.00 (7.44) 23.50 (6.96)*** Any Suicidal Behaviour       Strength Score      Vulnerability Score  7.86 (6.52)* 23.57 (6.13)  3.50 (1.73)*** 28.50 (6.14)** Any Self-Neglect       Strength Score      Vulnerability Score  12.45** 19.98*  11.77** 18.40 Any Unauthorized Leave       Strength Score      Vulnerability Score  12.43 (6.09) 19.70 (6.50)  13.21 (6.28) 19.50 (7.84) Any Substance Abuse      Strength Score      Vulnerability Score  13.19 (6.58) 21.00 (6.16)  12.52 (6.26) 16.52 (6.65) Any Victimization       Strength Score      Vulnerability Score  13.14 (7.14) 19.60 (6.26)  13.66 (6.91) 18.48 (7.80) Note. *p ≤ 0.05, **p ≤ 0.01. ***p ≤ 0.001  5.3.5 Relationship between Low vs. Moderate/High START Risk Estimates and the Adverse Outcomes The odds of a civil psychiatric inpatient in our study experiencing the relevant adverse outcome was significantly higher if the START assessor estimated the risk for that outcome to be moderate or high.  Similarly, forensic psychiatric inpatients rated moderate/high risk were at greater risk than patients rated low risk of experiencing all outcomes except suicidal behaviour, self-neglect, and victimization. According to standards in the field the ORs are large (an OR greater than or equal to 2.5 is generally taken to represent the lower limit of strong association between predictor and outcome; Fleiss, Williams, & Dubro, 1986). It is important to note, however, that although most of the odds ratios are statistically significant, the 95% confidence intervals are quite large.   95  Table 5.5  Proportion of Civil and Forensic Psychiatric Patients with Low vs. Moderate/High START Risk Estimates and the Odds Ratios for the presence of Adverse Outcomes:  A six month follow-up  Proportion of Patients Rated Moderate or High Risk who exhibited the outcome of concern  (N / %) /   Odds Ratio (95% CI) Outcome Civil Inpatients Forensic Inpatients Verbal Aggression              33 (94.3%) 7.41 (1.63-33.65) 33 (80.05%) 6.82 (2.69-17.27) Aggression Against Property                 24 (68.6%) 5.98 (2.47-14.49) 21 (51.2%) 6.07 (2.38-15.47) Aggression Against Others                 29 (82.9%) 8.89 (3.26-24.29) 26 (63.4%) 8.84 (3.49-22.39) Self-Harm                  4 (44.4%) 7.82 (1.78-34.46) 3 (60.00%) 11.73 (1.76-78.10) Suicidal Behaviour                  2 (40.0%) 12.80 (1.83-94.81) 1 (25.0%) 10.56 (0.83-133.61) Self-Neglect                  64 (94.1%) 11.64 (3.51-38.56) 25 (69.4%) 1.57 (0.66-3.73) Unauthorized Leave                  36 (85.7%) 10.00 (3.67-27.22) 8 (53.3%) 15.43 (4.16-57.19) Substance Abuse                 22 (71.0%) 43.39 (12.17-154.69) 21 (53.8%) 17.21 (5.22-56.70) Victimization   34 (85.0%) 9.07 (3.33-24.68) 9 (47.4%) 1.87 (0.68-5.13)  5.3.6 Incremental Validity of Risk Estimates over START Total Scores Table 6 presents results of logical regression testing the incremental utility of START risk estimates over START strength and vulnerability scores.  The results follow a clear pattern - for every outcome, the model including the relevant risk estimate out performs the model with total scores only.  In addition, when the relevant risk estimate was included, the model improvement is larger for the civil psychiatric cohort than the forensic cohort.  Model improvement with the addition of the relevant risk estimate was especially pronounced in the civil cohort for self-neglect, unauthorized leave, substance abuse, and victimization.  This pattern was also evident for unauthorized leave in the forensic cohort.   96  Table 5.6  Incremental Validity of Risk Estimates over START Total Scores  Step 1 Strength and Vulnerability Total Scores Step 2 Total Scores plus Risk Estimate B Wald Odds Ratio (CI) Model Fit B Wald Odds Ratio (CI) Model Fit Verbal Aggression (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (L vs. MH) Verbal Aggression (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (continuous)   0.01 0.08   0.12** 0.09*      0.02 3.06   9.87 6.11   1.01 (0.93-1.08) 1.08 (0.99-1.17)   1.12+(1.05-1.21) 1.09+(1.02-1.17)   χ2 (2)=4.20    χ2 (2)=32.23***   -0.01 0.07 1.93*  0.11* 0.06 1.23*   -0.01 0.07 1.75*  0.11** 0.05 1.23*   0.08 2.65 6.11  8.73 2.54 5.11   0.11 2.50 5.71  8.32 1.97 5.91   0.99 (0.92-1.07) 1.07 (0.99-1.17) 6.99 (1.49-31.92)  1.12+(1.04-1.20) 1.06 (0.99-1.15) 3.44(1.18-10.02)   0.99 (0.91-1.07) 1.07 (0.98-1.14) 5.75(1.37-24.15)  1.12+ (1.04-1.20) 1.06 (0.98-1.14) 3.43(1.27-9.24)   χ2 (3)=13.07**    χ2 (3)=37.52***     χ2 (3)=14.07**    χ2 (3)=38.88*** Property Aggression (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (L vs. MH) Property Aggression (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths   0.05 0.03   0.21*** 0.09*    2.18 0.82   16.15 4.46    1.05 (0.98-1.13) 1.03 (0.96-1.11)   1.23 (1.11-1.37) 1.09 (1.01-1.19)       χ2 (2)=5.28    χ2 (2)=41.55***   0.03 0.03 1.67***  0.20*** 0.05 0.83   0.02 0.02 1.42***  0.20***   0.70 0.57 13.12  14.34 0.89 2.77   0.38 0.32 13.76  14.34   1.03 (0.96-1.11) 1.03 (0.96-1.11) 5.32 (2.15-13.12)  1.23 (1.11-1.37) 1.05 (0.96-1.16) 2.70 (0.78-9.41)   1.02 (0.95-1.11) 1.02 (0.95-1.11) 4.13 (1.95-8.74)  1.23 (1.10-1.36)   χ2 (3)=19.35***    χ2 (3)=44.03***      χ2 (3)=22.31***  97   Step 1 Strength and Vulnerability Total Scores Step 2 Total Scores plus Risk Estimate B Wald Odds Ratio (CI) Model Fit B Wald Odds Ratio (CI) Model Fit           Vulnerabilities           Risk Estimate for Violence (continuous) 0.05 0.83 0.90 2.77 1.05 (0.95-1.15) 2.30 (0.86-6.13)    χ2 (3)=44.40*** Aggression Against Others (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (L vs. MH) Aggression Against Others (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Violence (continuous)   0.10** 0.01   0.07* 0.05             6.87 0.05   3.97 2.68             1.10 (1.03-1.18) 1.01 (0.94-1.08)   1.07 (1.00-1.15) 1.06 (0.99-1.13)             χ2 (2)=10.21**    χ2 (2)=13.32**              0.09* 0.00 2.05***  0.06 -0.01 1.96***   0.08* 0.00 1.86***  0.05 -0.02 1.82***   4.37 0.00 15.22  2.68 0.02 12.76   3.78 0.00 14.38  1.96 0.33 14.20   1.09 (1.01-1.18) 1.00 (0.93-1.08) 7.74 (2.77-21.65)  1.06 (0.99-1.15) 1.00 (0.92-1.08) 7.13 (2.43-20.94)   1.08 (1.00-1.17) 1.00 (0.92-1.08) 6.40 (2.45-16.71)  1.06 (0.98-1.14) 0.98 (0.90-1.06) 6.19 (2.40-15.99)   χ2 (3)=28.53***    χ2 (3)=27.40***     χ2 (3)=31.46***    χ2 (3)=30.81***    98   Step 1 Strength and Vulnerability Total Scores Step 2 Total Scores plus Risk Estimate B Wald Odds Ratio (CI) Model Fit B Wald Odds Ratio (CI) Model Fit Self-Harm (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Harm (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Harm (L vs. MH) Self-Harm (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Harm (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Harm (continuous)   0.03 -0.08   0.02 0.12*             0.45 2.50   0.18 5.10             1.03 (0.94-1.14) 0.92 (0.83-1.02)   1.02 (0.93-1.13) 1.13+(1.02-1.25)             χ2 (2)=2.57    χ2 (2)=10.07**              0.05 -0.10 2.26**  0.00 0.12* 2.20   0.04 -0.10 1.61*  0.01 0.11* 1.01   0.94 3.19 8.03  0.00 4.75 3.55   0.56 3.02 6.53  0.03 4.24 1.59   1.05 (0.95-1.17) 0.90 (0.80-1.01) 9.55 (2.01-45.50)  1.00 (0.90-1.11) 1.12(1.01-1.25) 9.02 (0.92-88.82)   1.04 (0.94-1.15) 0.91 (0.81-1.01) 5.03+(1.46-17.34)  1.01 (0.91-1.12) 1.12 (1.01-1.24) 2.75 (0.57-13.25)   χ2 (3)=9.95*    χ2 (3)=13.71**     χ2 (3)=9.14*    χ2 (3)=11.87**   Suicidal Behaviour (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Suicide(L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Suicide (L vs. MH) Suicidal Behaviour (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Suicide (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Suicide (continuous)   0.16 0.06   0.25 0.20             3.30 0.84   2.14 2.15             1.18 (0.99-1.40) 1.07 (0.93-1.22)   1.28 (0.92-1.77) 1.22 (0.93-1.60)             χ2 (2)=7.29*    χ2 (2)=13.61**             0.18* 0.11 3.64**  0.30 0.14 4.07   0.17 0.11 3.23*  0.30 0.14 4.07   3.90 1.80 7.70  2.25 1.14 1.38   3.05 1.91 5.45  2.25 1.14 1.38   1.20(1.00-1.44) 1.12 (0.95-1.31) 38.06 (2.91-497.84)  1.35 (0.91-2.00) 1.15 (0.89-1.48) 58.56 (0.07-52571.96)   1.18 (0.98-1.42) 1.12 (0.95-1.31) 25.19 (1.68-378.44)    χ2 (3)=14.65**    χ2 (3)=15.80**     χ2 (3)=15.36**     99   Step 1 Strength and Vulnerability Total Scores Step 2 Total Scores plus Risk Estimate B Wald Odds Ratio (CI) Model Fit B Wald Odds Ratio (CI) Model Fit  1.35 (0.91-2.00) 1.15 (0.89-1.48) 58.56 (0.07-52571.96) χ2 (3)=15.80**   Self-Neglect (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Neglect (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Neglect (L vs. MH) Self-Neglect (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Neglect (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Self-Neglect (continuous)   0.07 0.06   0.06 0.00             2.73 1.81   3.77 0.01             1.07 (0.99-1.16) 1.07 (0.98-1.17)   1.07 (1.00-1.13) 1.00 (0.94-1.07)             χ2 (2)=8.07*    χ2 (2)=5.65              0.08 0.00 2.37***  0.07* -0.01 0.41   0.08 0.01 1.74**  0.07 -0.01 0.53   3.10 0.05 13.36  3.94 0.03 0.77   3.00 0.01 11.01  3.88 0.06 1.65   1.09 (0.99-1.19) 1.00 (0.91-1.11) 10.73 (3.01-38.29)  1.07 (1.00-1.14) 1.00 (0.93-1.06) 1.51 (0.60-3.76)   1.08 (0.99-1.19) 1.01 (0.91-1.11) 5.71 (2.04-15.98)  1.07 (1.00-1.14) 0.99 (0.93-1.06) 1.69 (0.76-3.78)   χ2 (3)=24.46***    χ2 (3)=6.43     χ2 (3)=24.28***    χ2 (3)=7.40   Unauthorized Leave (UAL) (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for UAL (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for UAL (L vs. MH) Unauthorized Leave (UAL) (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities   0.05 0.00   -0.01 0.04         2.52 0.04   0.09 0.91         1.06 (0.99-1.13) 1.00 (0.93-1.07)   0.99 (0.90-1.08) 1.04 (0.96-1.14)         χ2 (2)=3.15    χ2 (2)=1.01          0.06 -0.05 2.39***  0.00 0.02 2.70***   0.06 -0.06 2.09***   2.59 1.24 20.05  0.00 0.14 15.98   2.47 1.97 19.07   1.06 (0.99-1.15) 0.96 (0.88-1.04) 10.92 (3.84-31.10)  1.00 (0.91-1.11) 1.02 (0.93-1.12) 14.90 (3.96-56.00)   1.06 (0.99-1.14) 0.94 (0.87-1.02) 8.06 (3.16-20.55)   χ2 (3)=28.93***    χ2 (3)=17.41**     χ2  100   Step 1 Strength and Vulnerability Total Scores Step 2 Total Scores plus Risk Estimate B Wald Odds Ratio (CI) Model Fit B Wald Odds Ratio (CI) Model Fit           Risk Estimate for UAL (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for UAL (continuous)                  0.00 0.02 2.70***  0.00 0.14 15.98  1.00 (0.91-1.11) 1.02 (0.93-1.12) 14.90 (3.96-56.00) (3)=34.35***    χ2 (3)=17.41**   Substance Abuse (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Substance Abuse (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Substance Abuse(L vs. MH) Substance Abuse (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Substance Abuse (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Substance Abuse (continuous)   -0.02 0.07   0.05 -0.05             0.32 2.66   2.17 2.09             0.98 (0.91-1.06) 1.07 (0.99-1.16)   1.05 (0.99-1.13) 0.95 (0.89-1.02)             χ2 (2)=2.84    χ2 (2)=2.89              0.03 -0.04 3.96***  0.06 -0.09* 3.13***   0.04 -0.09 2.98***  0.06 -0.10* 2.44***   0.40 0.46 29.90  2.06 4.63 22.03   0.65 1.94 25.38  1.86 5.02 22.74   1.04 (0.93-1.15) 0.96 (0.85-1.08) 52.51(12.69-217.19)  1.07 (0.98-1.16) 0.91 (0.84-0.99) 22.93 (6.20-84.81)   1.04 (0.94-1.16) 0.92 (0.81-1.04) 19.61(4.12-31.11)  1.06 (0.97-1.16) 0.90 (0.83-0.99) 11.43 (4.20-31.11)   χ2 (3)=50.11***    χ2 (3)=35.24***     χ2 (3)=48.92***    χ2 (3)=40.37***   Victimization (dichotomized RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Victimization (L vs. MH)      Forensic Psychiatric Inpatients           Inverted Strengths   0.00 0.02   -0.02 0.03   0.02 0.43   0.50 1.10   1.00 (0.93-1.06) 1.02 (0.96-1.10)   0.98 (0.92-1.04) 1.03 (0.97-1.10)   χ2 (2)=0.47    χ2 (2)=1.16    -0.02 0.03 2.22***  -0.02 0.03   0.25 0.45 18.57  0.58 0.74   0.98 (0.91-1.06) 1.03 (0.95-1.11) 9.23 (3.36-25.37)  0.98 (0.92-1.04) 1.03 (0.97-1.10)   χ2 (3)=2.34    χ2 (3)=24.02***  101   Step 1 Strength and Vulnerability Total Scores Step 2 Total Scores plus Risk Estimate B Wald Odds Ratio (CI) Model Fit B Wald Odds Ratio (CI) Model Fit           Vulnerabilities           Risk Estimate for Victimization(L vs. MH) Victimization (continuous RE)      Civil Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Victimization (continuous)      Forensic Psychiatric Inpatients           Inverted Strengths           Vulnerabilities           Risk Estimate for Victimization (continuous)                                         0.58   -0.02 0.03 2.16***  -0.03 0.02 0.58 1.19   0.36 0.50 18.12  0.61 0.51 2.47 1.79  (0.63-5.06)   0.98 (0.90-1.06) 1.03 (0.95-1.11) 8.68  (3.21-23.47)  0.98 (0.92-1.04) 1.02 (0.96-1.09) 1.79 (0.87-3.70)     χ2 (3)=25.00***    χ2 (3)=3.68   *p < 0.05, **p < 0.01. ***p ≤ 0.001        102  5.1.1 ROC Analyses of START Total Scores and Risk Estimates For forensic patients, the risk estimate for violence was predictive of verbal aggression, property aggression, and physical aggression.  The risk estimates for unauthorized leave and substance abuse among the forensic inpatients were also significantly predictive of those events occurring during the study’s six month follow-up.  For civil patients, the risk estimate for violence was again predictive of all three relevant outcomes:  verbal aggression; property aggression; and aggression against others.  In addition, the risk estimates for self-neglect, unauthorized leave, substance abuse, and victimization were also significantly associated with those events.  Forensic patients’ START vulnerability total scores were predictive of verbal aggression, property aggression, aggression against others, self-harm and suicidal behaviour.  In the civil psychiatric sample, START vulnerability total scores were only predictive of self-neglect.  Forensic inpatients’ START strength scores predicted the absence of verbal aggression, property aggression, aggression against others, suicidal behaviour, and self-neglect and, to a lesser extent, self-harm.  For the civil patients, the START strength scores predicted the absence of property aggression, aggression against others, suicidal behaviour, and self-neglect.    103  Table 5.7  Predicting Adverse Outcomes in Civil and Forensic Patients using START Assessments: ROC Analyses  Adverse Events / Sample Verbal Aggression Property Aggression Aggression Against Others Self-Harm Suicidal Behaviour Self-Neglect Unauthorized Leave Substance Abuse Victimization Violence Risk Estimate      Civil Psychiatric Inpatients      Forensic Inpatients  0.67 (0.01) 0.72 (0.00)  0.71 (0.00) 0.74 (0.00)  0.72 (0.00) 0.77 (0.00)       Self-Harm Risk Estimate      Civil Psychiatric Inpatients      Forensic Inpatients     0.63 (0.13) 0.60 (0.25)      Suicide Risk Estimate      Civil Psychiatric Inpatients      Forensic Inpatients      0.63 (0.25) 0.61 (0.46)     Self-Neglect Risk Estimate      Civil Psychiatric Inpatients      Forensic Inpatients       0.79 (0.00) 0.56 (0.30)    Unauthorized Leave Risk Estimate      Civil Psychiatric Inpatients      Forensic Inpatients        0.75 (0.00) 0.75 (0.00)   Substance-Abuse Risk Estimate       Civil Psychiatric Inpatients      Forensic Inpatients         0.88 (0.00) 0.83 (0.00)  Victimization Risk Estimate      Civil Psychiatric Inpatients      Forensic Inpatients          0.73 (0.00) 0.56 (0.35) Start Strength Score      Civil Psychiatric Inpatients      Forensic Inpatients  0.55 (0.43) 0.76 (0.00)  0.62 (0.04) 0.84 (0.00)  0.68 (0.00) 0.71 (0.00)  0.49 (0.93) 0.66 (0.06)  0.75 (0.03) 0.89 (0.01)  0.72 (0.00) 0.66 (0.01)  0.57 (0.21) 0.50 (0.98)  0.53 (0.67) 0.54 (0.57)  0.53 (0.61) 0.47 (0.62)  104  Adverse Events / Sample Verbal Aggression Property Aggression Aggression Against Others Self-Harm Suicidal Behaviour Self-Neglect Unauthorized Leave Substance Abuse Victimization Start Vulnerability Score      Civil Psychiatric Inpatients      Forensic Inpatients  0.60 (0.15) 0.75 (0.00)  0.56 (0.26) 0.76 (0.00)  0.56 (0.29) 0.66 (0.01)  0.36 (0.12) 0.75 (0.00)  0.69 (0.09) 0.88 (0.01)  0.66 (0.03) 0.58 (0.19)  0.54 (0.46) 0.57 (0.39)  0.60 (0.14) 0.45 (0.41)  0.52 (0.76) 0.55 (0.44) Cluster Membership      Civil Psychiatric Inpatients      Forensic Inpatients  0.55 (0.51) 0.71 (0.00)  0.51 (0.06) 0.68 (0.06)  0.56 (0.27) 0.58 (0.19)  0.43 (0.42) 0.66 (0.06)  0.64 (0.21) 0.76 (0.09)  0.59 (0.06) 0.71 (0.00)  0.56 (0.34) 0.54 (0.67)  0.55 (0.67) 0.40 (0.14)  0.54 (0.53) 0.54 (0.56)  105  5.1.2 Two-Step Cluster Analysis The file was first split to allow two separate models – one for the civil and one for forensic inpatients.   Both models had good average silhouette scores of 0.6 (a measure of cohesion and separation ranging from -1 to +1) and created groups that made sense from a practical, clinical perspective.  The civil inpatient model was a two cluster solution with cluster one (26.4% of sample) having lower vulnerability scores (M = 12.96) and higher strength scores (M = 21.43).  Cluster two (73.6% of sample) had higher vulnerability scores (M = 21.58) and lower strength scores (M = 10.34).  The forensic models were similar, but with very different distributions in the population.  Cluster one (51.0% of sample) had higher vulnerability scores (M = 24.06) and lower strength scores (M = 9.71).  Cluster two (49.0% of sample) had lower vulnerability scores (M = 10.93) and higher strength scores (M = 17.19).  Cluster membership was not predictive for the civil psychiatric group.  In the forensic cohort, cluster membership was predictive of verbal aggression, sexually inappropriate behaviour and self-neglect. 5.2 Discussion Our study found significant differences between the mean strength and vulnerability scores for both civil and forensic inpatients who presented with or without the various adverse outcomes measured in the present study.  This indicates there is a relationship between the actuarial scores and the outcomes.  Although actuarial strength and vulnerability scores (which are only totaled for research purposes) predicted many of the outcomes, the structured professional judgment risk estimates were more strongly predictive.  This was particularly true for the civil psychiatric cohort.  Gray et al. (2011) similarly found that the professional risk estimates out-performed actuarial strength scores in a civil psychiatric sample.  It may be that although the START items and actuarial scores are important, as evidenced by the relationship  106  between the scores and the outcomes, the importance of professional clinical judgement is especially critical for civil psychiatric inpatients.  Additionally, most of the odds ratios considering risk estimates dichotomized into low vs. moderate/high risk were significant.  In summary, START worked as it was intended to function, as and SPJ measure, and performed well for both civil and forensic inpatients across a diverse range of important patient safety outcomes.  The results from this study diverged in some important ways from the studies highlighted in Table 1.  For example, Braithwaite et al. (2010) found that risk estimates predicted poorly, however, the strength and vulnerability scores were significantly predictive.  This divergent result may be related to the different type of START assessors used.  Both Gray et al. (2011) and the current study used STARTs completed for research purposes.  Braithwaite et al. (2010) used STARTs completed by clinicians for clinical purposes.  An important topic for further research would be to consider whether or not clinicians and research assistants process information and make decisions about risk in different ways.  Information gathered from such research could be utilized for training in order to maximize the decision making strengths of both clinicians and research assistants.  The divergent results might also be explained by the differences in sample sizes.  Also of note, even though our sample size was twice that of the other studies, several of our confidence intervals were quite large.  It will be important for future research to consider the relationship between potential sources of signal noise (i.e. the samples represented diverse patient groups, it is possible that the relationship between START assessments and the outcomes is different for different groups of patients such as open vs. closed wards), sample size and effect size.  107  Several researchers have investigated the incremental validity of START strength and/or vulnerability scores and have found divergent results (e.g. Wilson et al., 2010; Braithwaithe et al., 2010).  An inability to definitively demonstrate the incremental validity of strengths and/or vulnerabilities in the same statistical model is crucially limited by multicolinearity.  Strength and vulnerability scores were not intended to function as mirror images of each other.  However, a strength item score and the reciprocal vulnerability item scores are often closely related.  For example, if someone fails to demonstrate strengths on the Occupational item, they are also likely to have considerable vulnerabilities for the Occupational item.  This is not always the case. For example, a patient might be developing strengths in substance use (e.g. seeking help); however, their substance use might still result in serious negative consequences (i.e. substantial vulnerabilities).  Creating a cluster model including strengths and vulnerabilities was proposed as a possible solution to this problem.  The cluster model using START strength and vulnerability scores was predictive for forensic patients; however, they were not predictive for the civil psychiatric inpatients indicating that this may not be a complete solution. In this study STARTs were coded by research assistants.  This limits generalizability to clinical practice in civil and forensic settings.  In addition, risk management strategies were not recorded.  This may be an important factor in explaining the differences in predictive accuracy between the two populations in our study.  For sample, the forensic hospital is more attuned to risk assessment and management.  An important area of future research is to examine the link between risk assessment, risk management and risk prevention rather than focusing on predictive accuracy.  As a consequence we are limited in our ability to understand whether any non-significant predictive findings are a result of successful risk management or a failure of the instrument.  Additionally, although the total scores from the START assessments did not  108  consistently distinguish patients with or without adverse outcomes, this is not the intended use of the assessment. Consistent with the Structured Professional Judgment (SPJ) model, the authors of the START do not recommend relying on total scores, rather assessors complete risk estimates for each patient safety concern (low, moderate, high).  It is these SPJ risk estimates that are used to guide risk management practices and treatment plans (Webster et al, 2009).   109  Chapter 6:  Discussion 6.1 Realignment of Tertiary Psychiatric Services in British Columbia: Has this Province’s Redevelopment of Services Avoided the Negative Outcomes Historically Associated with Hospital Closures? In our evaluation of the RVH closure (see Chapter 2), we were able to determine the location of all initial study participants at each follow-up for the entire length of the study.  We are confident that only one participant spent any time residing in a correctional facility, one participant in a forensic hospital, and only one participant spent a small amount of time homeless.  These results echo findings from a recent administrative data-based study which also demonstrated the realignment of psychiatric services in British Columbia did not result in transinstitutionalization (Livingston et al., 2011).  A research study in Quebec found similar results with only two participants requiring care at a forensic hospital, no participants residing in jails or prisons, and two participants lost to follow-up who were suspected to be homeless (Lesage et al., 2000).  Similar results have also been replicated in other countries at the vanguard of deinstitutionalization.  For example, the TAPS Project found that in five years of follow-up, four participants became homeless (Leff et al., 2000), and three were incarcerated in prison (Trieman et al, 1998).  In sum, when best practices for the devolution of psychiatric services are followed, transinstitutionalization and homelessness can largely be avoided.  Moreover, patients’ mental health does not necessarily suffer and, in fact, several positive outcomes have been demonstrated in this study (e.g. improvements in independent living skills and perceived quality of life) and prior research (e.g. Leff & Trieman, 2000; McGrew et al., 1999; Trauer et al., 2001). It is important to note that the closure of Riverview Hospital (RVH) was not really a true example of deinstitutionalization.  It was a bed for bed transfer of services from one centralized  110  location to locations distributed throughout the province.  This might explain the lack of transinstitutionalization evidenced in this study.  The majority of our participants remained in facilities that provided 24 hour care (~70%).  However, a slow movement towards smaller facilities with less onsite programming was evidenced.  A similar trend has been reported elsewhere in the literature.  For example, the TAPS project found that most of their participants required facilities with the same level of support and there was only a slight trend for movement towards less supervised facilities (Trieman et al., 1998). In addition to the positive evidence regarding the lack of transinstitutionalization, the clinical and psychosocial outcomes demonstrated in this study parallel results from similar studies in industrialized/resource-rich countries.  For instance, consistent with findings in Indiana and Australia, we found no change in clinical symptoms and a significant increase in participant perceived quality of life (McGrew et al., 1999; Hobbs et al., 2002).  Additionally, participants evidenced decreases in some domains associated with the negative symptoms of schizophrenia including improved independent living skills and even some reduction in cognitive deficits.  These results support recovery models advocating that even long-stay patients can benefit from well executed psychosocial rehabilitation programs (Briand et al., 2006).  In summary, caring for severely mentally ill individuals in smaller-home like facilities in communities closer to patient’s communities of origin has therapeutic value without creating a problem with transinstitutionalization.   6.2 Risk Profiles, Risk Assessment and Patient Safety in Forensic and Civil Psychiatric Patients Chapter three investigated the risk profiles and patient safety outcomes of the RVH redevelopment patients, comparing and contrasting them with a sample of forensic psychiatric  111  inpatients.  We found significant differences between these two samples on a number of static, historical, and criminogenic factors. Results indicated that a preliminary examination of the presenting profile of the forensic inpatients reveals a higher risk cohort than the civil inpatients.  In contrast, the civil inpatients tended to present with more persistent and debilitating mental disorder, as indicated by a greater number of hospitalizations, lengthier index admissions and greater prevalence and severity of self-neglect.  These results are not surprising and are generally consistent with prior research.  For example, Dumont and colleagues (2012) found diagnosis with a psychotic disorder to be associated with long-term psychiatric hospitalization and significant criminal history associated with continued forensic hospitalization.  Nevertheless, a more thorough examination of the patients’ risks and needs using comprehensive START assessments suggested the two groups were comparable in many other respects.  In contrast to the static factors, the dynamic START risk profiles of the two groups were remarkably similar.  For instance, there were no significant differences between the two cohorts on four of the seven risk estimates and on START strength and vulnerability total scores.  Despite the historical risk profiles of the forensic inpatients, and consistent with the dynamic START assessments, the civil inpatients evidenced a similar or greater prevalence of many negative outcomes, including: verbal aggression, aggression against others, inappropriate sexual behaviour, suicidal ideation, self-neglect, unauthorized leave, and victimization.  It is also important to note that significantly fewer civil psychiatric inpatients than forensic psychiatric inpatients experienced did not experience any negative outcomes during the six-month follow-up period.  Civil psychiatric patients also experienced many of the negative outcomes in a greater variety and severity than the forensic psychiatric patients.    112  Forensic inpatients were found to be significantly less likely than civil inpatients to have a therapeutic alliance with a caregiver or have a positive peer in their support network providing support. These findings have important implications for risk management and treatment planning.  For example, patients who experience a therapeutic alliance have been found to demonstrate a more positive course while in hospital and better outcomes post-hospitalization (Hewitt & Coffey, 2005).  Therefore, despite the challenges inherent in working with patients in a forensic environment, it might be helpful for forensic practitioners to introduce some strategies introduced by civil psychiatric practitioners, such as the recovery model, albeit in a modified fashion. Previous research has also supported the importance of dynamic factors.  By way of an illustration, De Vries Robbé, de Vogal, Douglas and Nijman (2015) demonstrated that dynamic positive changes in risk and protective factors were associated reductions in violent recidivism. Consistent with the findings of  prior START research (e.g., Wilson et al., 2013), these findings lend support for the value and importance of integrating dynamic variables into risk assessments and treatment and management planning, rather than relying on static and historical information alone.   6.3 Utility of START, in vivo, in a Forensic Setting  A recent meta-analysis of the START literature demonstrated the dearth of studies examining the use of START in vivo.  Chapter 4 relied on START assessments by actual clinical teams both adding to our understanding of the psychometric properties of START in practice and increasing the generalizability of findings to clinical care settings.  Results indicated START strength scores and the vulnerability scores maintained good structural reliability when implemented into practice in this forensic service.  In addition, we found excellent dispersion  113  across the total range of item (0, 1, or 2) and scale scores (0-40).  This is important because the vast majority of the population of the BC forensic hospital are young males, many of whom have a documented history of long-term criminal justice involvement, mental health problems, substance abuse, poor occupational and educational attainment (see Nicholls et al., 2009).  For these reasons, it is particularly promising to see considerable dispersion on the START items, suggesting that a dynamic snapshot of the client’s current presentation on the START might be a useful means of distinguishing clients’ risk levels and informing appropriate risk management and treatment planning efforts in the short-term even among a sample that might be quite homogeneous in many respects. START total strengths scores, total vulnerabilities scores and item scores varied by setting and security level in the manner expected, that is, patients in the maximum security units had the highest vulnerabilities scores and lowest strength scores.  The patients in the medium security units had scores that fell between those of patients in the maximum units and minimum units.  Patients on minimum secure units demonstrated the highest mean strength scores and the lowest mean vulnerability scores.  In sum, the differences across the hospital security levels show that the more privileges, freedoms, and community access enjoyed by the patient the lower their vulnerabilities score and the higher the strengths score.  Although preliminary, the data potentially suggest that the START may be a useful method for monitoring treatment progress and making recommendations for placement and privileges. A somewhat unique aspect of START, compared to the pre-existing SPJ measures, is the consideration of mental health and risk for seven adverse events commonly seen in individuals living with mental illness and/or among institutionalized and marginalized populations (see Webster et al., 2006, 2009).  Results indicated there is significant overlap across the seven  114  START risk estimates.  For example, the treatment teams’ estimates of the risk of violence was significantly correlated with their determinations of the other risk estimates (i.e., self-harm, suicide, unauthorized leave, substance abuse, self-neglect, victimization).  This finding is supported by a literature review which found strong correlations between schizophrenia and multiple adverse outcomes (Kooyman, Dean, Harvey and Walsh, 2007), and supports the importance considering risk assessment, risk management and treatment planning for multiple co-occurring adverse patient safety outcomes (Hillbrand, 2001).  In sum, the results of the present study contribute to the developing body of evidence that suggests the START could provide insights into the risk management and treatment needs of forensic psychiatric inpatients.   6.4 Utility of START in a Civil and a Forensic Setting Considering Multiple Patient Safety Outcomes  Although the publication of the MacArthur Study in 2001 encouraged many individuals in the risk assessment field to consider the use to these tools in civil psychiatric populations (Webster, 2011), to date there has been only been limited permeation into civil psychiatric practice (O’Shea & Dickens, 2014; Crocker et al., 2011).  For example, there have only been two published research studies considering START in a civil psychiatric sample.  Chapter five found significant differences between the mean START strength and vulnerability scores for both civil and forensic inpatients that presented with or without the various adverse outcomes measured in the present study.  This indicates there is a relationship between the actuarial scores and the outcomes for both samples.  Although actuarial strength and vulnerability scores (which are only totalled for research purposes) predicted many of the outcomes, the structured professional judgment risk estimates were more strongly predictive.  This was particularly true for the civil  115  psychiatric cohort.  Gray et al. (2011) similarly found that the professional risk estimates out-performed actuarial strength scores in a civil psychiatric sample.  It may be that although the START items and actuarial scores are important, as evidenced by the relationship between the scores and the outcomes, the importance of professional clinical judgement is especially critical for civil psychiatric inpatients.  Additionally, most of the odds ratios considering risk estimates dichotomized into low vs. moderate/high risk were significant.  These finding are not surprising; findings from a recent international study of violence risk assessment highlighted the importance of professional judgment risk estimates in SPJ measures such as START (Nicholls et. al., 2015).  Summary risk judgments were articulated as important to guide placement decisions, conditions or privileges, and the frequency and duration of treatment (Nicholls et. al., 2015).   In summary, START worked as it was intended to function, as an SPJ measure, and performed well for both civil and forensic inpatients across a diverse range of important patient safety outcomes.  The findings from chapter five also demonstrate the extent to which risk assessment and treatment planning needs in civil psychiatry overlap with current practices in the forensic setting and highlight the importance of ensuring that the expertise in violence risk assessment and risk management commonly of focus in forensic services is also integrated into civil psychiatric care practices. The projection of forensic expertise ‘upstream’ into civil psychiatric training programs and treatment settings may provide a means of preventing adverse events that may result in serious harm to the patient, the public and/or the criminalization of mental illness (e.g., Crocker et al., 2015). 6.5 Methodological Considerations:  Strengths and Limitations The primary methodological strengths and limitations underlining the first study in this dissertation (Chapter two:  Redevelopment of Tertiary Psychiatric Services in BC) stem from the  116  challenges inherent in studying a real-life event.  The Riverview redevelopment project evolved in real time; included a severely disabled population; and covered a geographically expansive province.  One cohort of patients who moved to a facility in the far North of British Columbia were only interviewed every second year due to financial realities.  A significant number of participants at each follow-up were either too ill to be interviewed or refused to be interviewed.  To address these limitations, we conducted analyses to examine possible systematic biases in the missing data and found none modifying the results.   Despite the these limitations the study had several strengths, including a 5-year follow-up that traced all initial long-stay inpatients and the use of standardised questionnaires with patients and staff to demonstrate the effectiveness of a well-planned and resourced deinstitutionalization project.   The most significant limitations associated with the second study (chapter three), comparing characteristics of civil and forensic inpatients, related to sampling methodology.  The forensic sample included all current inpatients with an up-to-date START on file during the study time frame.  Completing STARTs within one month of admission and every three months thereafter for all treatment patients is policy at the forensic hospital.  Audited compliance rates indicate that during the study period 80-90% of patients had up-to-date STARTs on file.  In contrast, for the civil cohort, those inpatients whose psychiatrist deemed too ill to participate in the study were excluded. This means that some of the most difficult to manage civil inpatients were not included, indicating that our results may well underestimate the prevalence and severity of negative outcomes among tertiary civil psychiatric inpatients.  Additionally, we collected information on the prevalence of outcomes from multidisciplinary progress notes.  It is possible that care providers recorded adverse events differently from one setting to another and/or that events are not coded reliably (see Ehmann et al., 2001).  117  Our examination of START in a forensic setting (chapter four) was limited because we were unable to comment on the participants’ characteristics or risk factors that were not explicitly delineated on the START.  For instance, we were not able to report on prior hospitalizations/mental health history, the nature of the index offence, or prior contact with the criminal justice and forensic systems.  In addition, as a consequence of our reliance on secondary data we were unable to thoroughly test the field reliability of the measure, or the validity of START (predictive, incremental, convergent, and divergent). The comparison of the psychometric properties of START in forensic and civil psychiatric patients (chapter five) was limited as a result of STARTs being coded by research assistants.  This limits generalizability to clinical practice in civil and forensic settings.  In addition, risk management strategies were not recorded.  This may be an important factor in explaining the differences in predictive accuracy between the two populations in our study.  For example, the forensic hospital is more attuned to risk assessment and management.  As a consequence we are limited in our ability to understand whether any non-significant predictive findings are a result of successful risk management or a failure of the instrument.   6.6 Summation and Future Directions Taken together, the results of the studies that comprise this dissertation: 1) provide evidence that well-planned and resourced closures of civil psychiatric hospitals do not lead to negative outcomes such as transinstitutionalization and homelessness and in fact lead to positive results such as increases in quality of life and independent living skills; 2) there is significant overlap between civil psychiatric populations in British Columbia, especially in terms of dynamic variables and the prevalence of negative outcomes, highlighting the important of risk assessment and risk management in both populations; 3) START demonstrates strong  118  psychometric properties in clinical use in a forensic setting and shows promise as a tool to inform risk management, patient care, changes in privileges, and the consideration of changes in care environments (i.e. moving to a facility with fewer restrictions); and 4) START exhibits strong psychometric properties in both a civil and a forensic setting for multiple inter-related patient safety outcomes (violence, suicide, self-harm, self-neglect, unauthorized absence, substance use, and victimization). The generally promising results evidenced in this dissertation need to be considered in light of the limitations of the research design and associated caveats.  We studied a relatively small sample (N = 189 and N = 106 of approximately 500) of Riverview patients who left the hospital during a time of intense government and public scrutiny. It can be assumed that the planning and resources put into transitioning this group into the community would have positively influenced their experiences and outcomes.  This body of work does not provide any insight into the implications of the bed closures at the hospital during the 1960s, when Riverview started downsizing, until 2002, when the closure plans for Riverview were initiated.  Moreover, we cannot speak to what is happening and has happened to individuals in need of tertiary psychiatric services after the closure of Riverview. For example, how easy or difficult is if for an individual requiring tertiary care that would have historically been provided at Riverview, to access appropriate.  Examining the experiences of new consumers of mental health services today and into the future will be an important area of further research. The limitations of the generalizability of the study also extend to the methodology and context of the studies themselves.  We were not able to access all patients at Riverview during the final stages of the hospital closure.  This reflected pragmatic and ethical challenges.  For  119  instance, we required a psychiatrist’s assent before we approached any patient.  Therefore, we were not permitted to recruit some of the most severely disabled individuals into the study; for example, elderly patients suffering from dementia or patients for whom discussing the move was considered to be potentially too distressing.  This biased selection process means that our results can only fairly be generalized to individuals well enough to participate in the consent process.  Furthermore, there were a proportion of patients who consented to participate, but whom themselves or their psychiatrist deemed participating in the interview too onerous.  For these patients we have only file-based data.  We were similarly not able to measure all relevant aspects of the closure of Riverview.  For example, it could be possible that moving from an aging facility like Riverview to newly built or renovated facilities had a positive impact on staff moral, which in turn had a positive impact on patient’s outcomes.  Future research of deinstitutionalizations should consider the inclusion of additional measures to assess the impact of the transition on staff. Additional future directions for research emanating from this dissertation, both planned and ongoing, can be more or less divided into those involving the ongoing redevelopment of tertiary care in BC and those pertaining to START.  The redevelopment data presented in this dissertation is only a small part of a large-scale study investigating this important shift in psychiatric care in BC.  Forthcoming analyses of existing data will explore the facility and clinical characteristics that predict both positive and negative outcomes to examine whether resources were used efficiently.  We also plan to obtain budgetary information from each of the five Health Authorities in order to explore the relationship between cost efficiency and clinical effectiveness.  The BC Ministry of Health is continuing to strive to provide the best possible care  120  for individual experiencing mental illness.  For example, the ministry recently identified a need to further develop services for individuals with severe addition and mental illness.  They have plans to reconfigure emergency room services to better meet the needs of this population, increase the number of Assertive Outreach Teams, and increase services at the Burnably Centre for Mental Health and Addictions (BC Ministry of Health, 2015).  It is important to continue to evaluate the impact of these changes in service delivery.   The results from our START papers have motivated some in progress or planned research.  For example we are in the process of preparing for submission a paper investigating the field reliability of START and comparing and contrasting clinician completed and research assistant completed STARTs.  An important topic for further research would be to consider whether or not clinicians and research assistants process information and make decisions about risk in different ways.  Information gathered from such research could be utilized for training in order to maximize the decision making strengths of both clinicians and research assistants.  We also found that even though our sample size was twice that of the other studies, several of our confidence intervals were quite large.  It will be important for future research to consider the relationship between potential sources of signal noise (i.e. the samples represented diverse patient groups, it is possible that the relationship between START assessments and the outcomes is different for different groups of patients such as open vs. closed wards), sample size and effect size.  Another important area of future research is to examine the link between risk assessment, risk management and risk prevention rather than focusing on predictive accuracy.  We are hoping with changes in electronic reporting systems at the forensic hospital, in the future we will be able to more directly link information about risk assessments, risk management and treatments, with risk prevention.  121   At the intersection between deinstitutionalization and risk assessment, one consideration is if patients with certain historical risk profiles (i.e. violent, fire setters) are stigmatized in the system; which in turn may contribute to reinstitutionalization as a preventive measure.  It might be usefully to examine if dynamic risk assessments could reduce the stigma of these historical risk profiles; allowing patients to be treated in the best possible environment for their recovery.    122  References APA Presidential Task Force on Evidence-Based Practice. (2006). Evidence-based practice in psychology.  American Psychologist, 61, 271-285. Abidin, Z., Davoren, M., Naughton, L., Gibbons, O., Nulty, A., & Kennedy, H.G. (2013).  Susceptibility (risk and protective) factors for in-patient violence and self-harm: prospective study of structured professional judgment instruments START and SAPROF, DUNDRUM-3 and DUNDRUM-4 in forensic mental health services.  BMC Psychiatry, 13, 197-215. doi: 10.1186/1471-244X-13-197 Alberta Health. (2015). Creating Connections:  Alberta’s Addiction and Mental Health Strategy Implementation:  Interim Report 2011-2014.  Retrieved from:  http://www.health.alberta.ca/documents/Mental-Health-Creating-Connections-Report-2015.pdf. Aviram U, Minsky S, Smoyak S, Gubman-Riesser G., 1995.  Discharge-ready patients who remain hospitalized: A re-emerging problem for mental health services.  Psychiatr. Q.  66 (1) 63-85. Bachrach L., 2001.  The state of the state mental hospital at the turn of the century.  New Dir. For Ment. Health Serv.  Fall (91) 89-106. Barbato A., 1998.  Psychiatry in transition: Outcomes of mental health policy shift in Italy. Aust. N. Z. J. Psychiatry.  32 (5) 673-679. Becker M., 1998.  A US experience: consumer responsive quality of life measurement.  Can. J. Commun. Ment. Health.  (3 Suppl) 41. Borum, R., Bartel, P., & Forth, A. (2006). Manual for the Structured Assessment for Violence Risk in Youth (SAVRY). Odessa, FL: Psychological Assessment Resources.   123  Borum, R., & Reddy, M. (2001). Assessing violence risk in Tarasoff situations: A fact-based model of inquiry. Behavioral Sciences and the Law, 19, 375-385. Braithwaite, E., Charette, Y., Crocker, A.G. & Reyes, A. (2010). The predictive validity of clinical ratings of the Short-Term Assessment of Risk and Treatability (START). International Journal of Forensic Mental Health Services, 9, 271 – 281. doi: 10.1080/14999013.2010.534378 Briand C, Vasiliadis HM, Lesage AD,et al, 2006.  Including integrated psychological treatment as part of standard medical therapy for patients with schizophrenia: clinical outcomes.  J. Nerv. Ment. Dis.  194 (7) 463-470. British Columbia Government.  Quick Facts about BC, 2012.    Retrieved August 1, 2012 from http://www.bcstats.gov.bc.ca/Publications/QuickFactsAboutBC.aspx   British Columbia Mental Health and Addiction Services.  Riverview Redevelopment Project.  Retrieved August 2, 2010, from http://www.bcmhas.ca/AboutUs/News/ Riverview_Redevelopment/default.htm British Columbia Mental Health and Addiction Services, 2010.  History – BC Mental Health Timeline.  Retrieved August 2, 2010, from http://www.bcmhas.ca/AboutUs/History.htm British Columbia Mental Health and Addiction Services, 2009.  From http://www.bcmhas.ca/ AboutUs/ News / Riverview_Redevelopment /RedevForum06QA.htm British Columbia Ministry of Health (2013).  Improving Health Services for Individuals with Severe Addition and Mental Illness.  Retrieved from:  http://www.health.gov.bc.ca/library/publications/year/2013/improving-severe-addiction-and-mental-illness-services.pdf  124  British Columbia Ministry of Health, 1998.  The 1998 Mental Health Plan – Revitalization and Rebalancing British Columbia’s Mental Health System.  Retrieved August 2, 2010, from http://www.health.gov.bc.ca/library/ publications /year/1998/Mental_Health_Plan.pdf BC Mental Health & Addiction Services (2014).  Riverview Hospital Redevelopment Project Plan:  Summary of Replacement Bed Allocations, February 2012.  Coquitlam:  Author. Bryeton, L., 2006, November 2.  Creating a community where everyone belongs.  The Vancouver Sun, p. A19.   Burns, T. (2014). Community psychiatry's achievements. Epidemiology And Psychiatric Sciences, 23(4), 337-344. doi:10.1017/S2045796014000560 Campbell, M. B., French, S., & Gendreau, P. (2009). The prediction of violence in adult offenders: A meta-analytic comparison of instruments and methods of assessment. Criminal Justice and Behavior, 36, 567-590. [doi: 10.1177/0093854809333610] Chaimowitz, G. (2012). The criminalization of people with mental illness. The Canadian Journal Of Psychiatry / La Revue Canadienne De Psychiatrie, 57(2).  Charette, Y., Crocker, A. G., & Billette, I. (2014). Police encounters involving citizens with mental illness: Use of resources and outcomes. Psychiatric Services, 65(4), 511-516. doi:10.1176/appi.ps.201300053 Cooper, B. S., Griesel, D., & Yuille, J. C. (2007). Clinical-Forensic Risk Assessment: The Past and Current State of Affairs. Journal Of Forensic Psychology Practice, 7(4), 1-63. doi:10.1300/J1 58v07n04-01  125  Côté, G., Lesage, A., Chawky, N., & Loyer, M. (1997). Clinical specificity of prison inmates with severe mental disorders: A case-control study. The British Journal Of Psychiatry, 170571-577. doi:10.1192/bjp.170.6.571 Crocker, A. G., Braithwaite, E., Laferrière, D., Gagnon, D., Venegas, C., & Jenkins, T. (2011). START changing practice: Implementing a risk assessment and management tool in a civil psychiatric setting. International Journal of Forensic Mental Health, 10, 13-28. [doi: 10.1080/14999013.2011.553146] Crocker, A.G., Nicholls, T.L., Seto, M.C., Côté, G., Charette, Y., & Caulet, M. (2015). The National Trajectory Project of individuals found Not Criminally Responsible on account of Mental Disorder in Canada. Part 1: Context and Methods. Canadian Journal of Psychiatry, 60(3), 98-105. Crocker, A.G., Nicholls, T.L., Seto, M.C., Côté, G., Charette, Y., & Caulet, M. (2015). The National Trajectory Project of individuals found Not Criminally Responsible on account of Mental Disorder in Canada. Part 1: Context and Methods. Canadian Journal of Psychiatry, 60(3), 98-105. Cyr M, Toupin J, Lesage A, Valiquette C, 1994.  Assessment of independent living skills for psychotic patients: Further validity and reliability.  J. Nerv. Ment. Dis.  182 (2) 91-97. Dayson, D., 1993.  The TAPS Project. 12: Crime, vagrancy, death and readmission of the long-term mentally ill during their first year of local reprovision.  Br. J. Psychiatry Suppl.  19 40-44. D'Avanzo B, Barbato A, Barbui C, Battino R, Civenti G, Frattura L. (2003).  Discharges of patients from public psychiatric hospitals in Italy between 1994 and 2000.  Int. J. Soc. Psychiatry.  49 (1) 27-34.  126  De Vries Robbé, M., de Vogel, V., Douglas, K. S., & Nijman, H. I. (2015). Changes in dynamic risk and protective factors for violence during inpatient forensic psychiatric treatment: Predicting reductions in postdischarge community recidivism. Law And Human Behavior, 39(1), 53-61. doi:10.1037/lhb0000089 Department of Health. (2007). Best practice in managing risk: Principles and evidence for best practice in the assessment and management of risk to self and others in mental health services. London: Department of Health, United Kingdom. Desmarais, S. L., Nicholls, T. L., & Brink, J. (2007, June). User satisfaction and predictive validity of file-based short-term assessments of risks: Comparisons with treatment team assessments. In M. Doyle (Chair), Assessing violence risks in the short-term: Refining START through research, clinical practice, and conceptual clarification. Symposium presented at the 7th annual conference of the International Association of Forensic Mental Health Services, Montreal, QC, Canada. Desmarais, S. L., Nicholls, T. L., Read, J. D., & Brink, J. (2010). Confidence and accuracy in short-term assessments of risk across overlapping domains. Journal of Forensic Psychiatry & Psychology, 21(1), 1-22. [doi: 10.1080/14789940903183932] Desmarais, S., Nicholls, T. L., Wilson, C., & Brink, J. (2011). Reliability and validity of the Short-Term Assessment of Risk and Treatability in Assessing Risk for Inpatient Aggression. Manuscript submitted for publication. Diamond R, Becker M., 1999.  The Wisconsin Quality of Life Index: A multidimensional model for measuring quality of life.  J. Clin. Psychiatry.  60 (Suppl 3) 29-31.  127  Doyle, M., Lewis, G. & Brisbane, M. (2008). Implementing the Short-Term Assessment of Risk and Treatability (START) in a forensic mental health service. Psychiatric Bulletin, 32, 406-408. [doi:10.1192/pb.bp.108.019794]    Dumais, A., Côté, G., Larue, C., Goulet, M., & Pelletier, J. (2014). Clinical characteristics and service use of incarcerated males with severe mental disorders: A comparative case-control study with patients found not criminally responsible. Issues In Mental Health Nursing, 35(8), 597-603. doi:10.3109/01612840.2013.861885 Dumont, M., Dumais, A., Briand, C., Côté, G., Lesage, A., & Dubreucq, J. (2012). Clinical characteristics of patients deemed to require long-term hospitalization in a civil or forensic psychiatric setting. The International Journal of Forensic Mental Health, 11(2), 110-118. doi:10.1080/14999013.2012.690019 Duurkoop P, van Dyck R., 2003.  From a 'State Mental Hospital' to new homes in the city: Longitudinal research into the use of intramural facilities by long-stay care-dependent psychiatric clients in Amsterdam.  Community Ment. Health.  39 (1) 77-92. Ehmann, T. S., Smith, G. N., Yamamoto, A., McCarthy, N., Ross, D., Au, T., & . Honer, W. G. (2001). Violence in treatment resistant psychotic inpatients. Journal of Nervous and Mental Disease, 189(10), 716-721. doi:10.1097/00005053-200110000-00009 Every-Palmer, S., & Norris, J. (2013). Not guilty by reason of epilepsy. Post-ictal delirium and psychosis resulting in violent offending. Australian And New Zealand Journal Of Psychiatry, 47(10), 961-962. doi:10.1177/0004867413479409 Fabisiak S, Baskin D, Hammer R., 1983.  The New York State Level of Care Survey in community mental health centers and New York State's psychiatric centers.  Psychiatr. Q.  55 (1) 25-34.  128  Fakhoury, W. & Priebe, S. (2007). Deinstitutionalization and rehabilitation: major changes in the provision of mental healthcare.  Psychiatry, 6(8), 313-316. Fleiss, J. L. (1981). Balanced incomplete block designs for inter-rater reliability studies. Applied Psychological Measurement, 5(1), 105-112. doi:10.1177/014662168100500115 Ford S., 1987.  Into the outside.  Nursing Times.  83 (20) 40-42. Garety P, Morris I., 1984.  A new unit for long-stay psychiatric patients: organization, attitudes and quality of care.  Psychol. Med.  14 (1) 183-192. Gray, N.S., Benson, R., Craig, R., Davies, H., Fitzgerald, S., Huckle, P., Maggs, R., Taylor, J., Trueman, M., Williams, T. & Snowden, R.J. (2011). The Short-Term Assessment of Risk and Treatability (START): A prospective study of inpatient behaviour.  International Journal of Forensic Mental Health, 10(3), 305-313. doi: 10.1080/14999013.2011.631692 Grindlay, L., 2009, May 7.  The long road out of Riverview; Moving the mentally ill back into their communities is proving a tough task.  The Province, p. A8. Gudeman J, Shore M., 1984.  Beyond deinstitutionalization. A new class of facilities for the mentally ill.  N. Engl. J. Med.  311 (13) 832-836. Guy, L. S. (2008). Performance indicators of the structured professional judgment approach for assessing risk for violence to others: A meta-analytic survey (Unpublished doctoral dissertation), Department of Psychology, Simon Fraser University, Burnaby, BC. Haley G, Iverson G, Moreau M., 2002.  Development of the Riverview Psychiatric Inventory.  Psychiatr. Q.  73 (3) 249-256. Haque, Q., Cree, A., Webster, C., & Hasnie, B. (2008). Best practice in managing violence and related risks. Psychiatric Bulletin, 32, 403-405. [doi: 10.1192/pb.bp.108.019927]  129  Hart, S. D., Cox, D. N., & Hare, R. D. (1995). The Hare Psychopathy Checklist-Revised: Screening Version. Toronto: Multi Health Systems. Heilbrun, K., Golloway, G. G., Shoukry, V. E., & Gustafson, D. (1995). Physical control of patients on an inpatient setting: Forensic vs. civil populations. Psychiatric Quarterly, 66(2), 133-145. doi:10.1007/BF02238860 Hewitt, J., & Coffey, M. (2005). Therapeutic working relationships with people with schizophrenia: Literature review. Journal Of Advanced Nursing, 52(5), 561-570. doi:10.1111/j.1365-2648.2005.03623.x Hillbrand, M. (2001). Homicide-suicide and other forms of co-occurring aggression against self and against others. Professional Psychology:  Research & Practice, 32, 626-635. Hobbs C, Tennant C, Brown J, et al., (2000).  Deinstitutionalisation for long-term mental illness: A 2-year clinical evaluation. Aust. N. Z. J. Psychiatry.  34 (3) 476-483. Hobbs C, Newton L, Tennant C, Rosen A, Tribe K., 2002.  Deinstitutionalization for long-term mental illness: a six-year evaluation.  Aust. N. Z. J. Psychiatry.  36 (1) 60-66. Institute of Health Economics (2014). Criminal justice and forensic psychiatry costs in Alberta. Retrieved from: http://www.ihe.ca/download/criminal_justice_and_forensic_psychiatry_costs_in_alberta_2014.pdf. Jackson, R. L., & Guyton, M. R. (2008). Violence Risk Assessment.  In R. L. Jackson (Ed.), Learning Forensic Assessment (pp. 153-181). New York, NY: Routledge. Jansman-Hart, E. M., Seto, M. C., Crocker, A. G., Nicholls, T. L., & Côté, G. (2011). International trends in demand for forensic mental health services. The International  130  Journal Of Forensic Mental Health, 10(4), 326-336. doi:10.1080/14999013.2011.625591 Johnson S, Kuhlmann R., 2000.  The European Service Mapping Schedule (ESMS): development of an instrument for the description and classification of mental health services.  Acta Psychiatr. Scand. 405 (Sup) 14-23. Jones D., 1993.  The TAPS Project. 11: The selection of patients for reprovision.  Br. J. Psychiatry Suppl.  19 36-39. Kirby, M. J. L. (2006) Out of the Shadows at Last: Transforming mental health, mental illness and addiction services in Canada.. The Standing Senate Committee on Social Affairs, Science and Technology. Ottawa, Canada. Accessed September 12, 2008 from http://www.parl.gc.ca/39/1/parlbus/commbus/senate/com-e/soci-e/rep-e/rep02may06-e.htm.160. Knapp M, Beecham J, McDaid D, Matosevic T, Smith M., 2011. The economic consequences of deinstitutionalisation of mental health services: lessons from a systematic review of European experience.  Health Soc. Care Community. 19 (2) 113-125. Kooyman, I., Dean, K., Harvey, S., Walsh, E. (2007). Outcomes of public concern in schizophrenia.  British Journal of Psychiatry, 191, s29-s36. Kroppan, E., Nesset, M. B., Nonstad, K., Pedersen, T. W., Almvik, R., & Palmstierna, T. (2011). Implementation of the Short Term Assessment of Risk and Treatability (START) in a forensic high secure unit. International Journal of Forensic Mental Health, 10, 7-12. [doi: 10.1080/14999013.2011.552368] Lamb H, Bachrach L., 2001.  Some perspectives on deinstitutionalization.  Psychiatr. Serv.  52 (8) 1039-1045.  131  Lambert E., 1982.  Reliability of subscales of the New York State Level of Care Survey.  J. Psychiatr. Treat. Eval.  4 (5) 427-431. Landgraf, S., Blumenauer, K., Osterheider, M., & Eisenbarth, H. (2013). A clinical and demographic comparison between a forensic and a general sample of female patients with schizophrenia. Psychiatry Research, 210(3), 1176-1183. doi:10.1016/j.psychres.2013.09.009 Landis, J.R.; Koch, G.G. (1977). "The measurement of observer agreement for categorical data". Biometrics 33 (1): 159–174. doi:10.2307/2529310 Lawrence D, Kisely S, Pais J., 2010.  The epidemiology of excess mortality in people with mental illness.  Can. J. Psychiatry.  55 (12) 752-760. Leff J, Trieman N, Knapp M, Hallam A., 2000.  The TAPS Project: A report on 13 years of research, 1985–1998.  Psychiatr. Bulletin.  24 (5) 165-168. Leff J, Trieman N., 2000.  Long-stay patients discharged from psychiatric hospitals: Social and clinical outcomes after 5 years in the community.  The TAPS project 46.  Br. J. Psychiatry.  176 217-223. Leff J, Trieman N, Gooch C., 1996.  Team for the Assessment of Psychiatric Services (TAPS) Project 33: prospective follow-up study of long-stay patients discharged from two psychiatric hospitals.  Am. J. Psychiatry.  153 (10) 1318-1324. Leff J, Thornicroft G, Coxhead N, Crawford C., (1994).  The TAPS Project. 22: A five-year follow-up of long-stay psychiatric patients discharged to the community.  Br. J. Psychiatry Suppl.  25 13-17.  132  Lesage, A. D., Groden, D., Goldner, E. M., Gélinas, D. & Arnold, L. M. Regionalised tertiary psychiatric residential facilities. Epidemiologia e psichiatria sociale Jan-Mar 2008;17(1):38-46. PMID:18444457 Lesage, A., Groden, D., Goldner, E. M., Gelinas, D., & Arnold, L. M. (2008). Regionalised tertiary psychiatric residential facilities. Epidemiologia E Psichiatria Sociale, 17(1), 38-46. Lesage A, Morissette R, Fortier L, Reinharz D, Contandriopoulos A. I., 2000.  Downsizing psychiatric hospitals: Needs for care and services of current and discharged long-stay inpatients. Can. J. Psychiatry.  45 (6) 526-531. Lesage A, Tansella M., 1993.  Comprehensive community care without long stay beds in mental hospitals: Trends from an Italian good practice area.  Can. J. Psychiatry.  38 (3) 187-194. Lesage A, Trapani V, Tansella M., 1990.  Excess mortality by natural causes of Italian schizophrenic patients.  Arch. Psychiatry Neurol. Sci.  239 (6) 361-365. Levaux M, Potvin S, Sepehry A, Sablier J, Mendrek A, Stip E., 2007.  Computerized assessment of cognition in schizophrenia: Promises and pitfalls of CANTAB.  Eur. Psychiatry.  22 (2) 104-115. Lincoln, T. M., Hodgins, S. S., Müller-Isberner, R. R., Jöckel, D. D., Freese, R. R., Born, P. P., & ... Gretenkord, L. L. (2005). Sind sie gefährlicher?--Entlassene Patienten des psychiatrischen Maßregelvollzuges und der Allgemeinpsychiatrie im Vergleich. = Is there Any Difference? The Risk of Violence in Forensic and General Psychiatric Patients During a Two Year Period After Discharge. Krankenhauspsychiatrie, 16(2), 48-56.   133  Livingston J, Nicholls T, Brink J., 2011.  The impact of realigning a tertiary psychiatric hospital in British Columbia on other institutional sectors.  Psychiatr. Serv.  62 (2) 200-205. Macfarlane D, Fortin P, Fox J, Gundry S, Oshry J, Warren E., 1997.  Clinical and human resource planning for the downsizing of psychiatric hospitals: the British Columbia experience.  Psychiatr. Q.  68 (1) 25-42. Malla A, Williams R, Kopala L, Smith G, Talling D, Balshaw R., 2006.  Outcome on quality of life in a Canadian national sample of patients with schizophrenia and related psychotic disorders.  Acta Psychiatr. Scand.  113 (Suppl 430) 22-28. McGrew J, Wright E, Pescosolido B, McDonel E., 1999 (a).  The closing of Central State Hospital: Long-term outcomes for persons with severe mental illness.  J. Behav. Health Serv. Res.  26 (3) 246-261. McGrew J, Wright E, Pescosolido B., 1999 (b).  Closing of a state hospital: An overview and framework for a case study.  J. Behav. Health Serv. Res..  26 (3) 236-245. Micale, M. S. (2014). The ten most important changes in psychiatry since World War II. History Of Psychiatry, 25(4), 485-491. doi:10.1177/0957154X14547460 Miller, H. A. (2006). A dynamic assessment of offender risk, needs, and strengths in a sample of general offenders. Behavioral Sciences and the Law, 24, 767-782. Morrow M, Dagg P, Pederson A., 2008.  Is deinstitutionalization a ‘failed experiment’? The ethics of re-institutionalization. J. Ethics Mental Health.  3 (2) 1-7. Newton L, Rosen A, Tennant C, Hobbs C, Lapsley H, Tribe K., 2000.  Deinstitutionalisation for long-term mental illness: An ethnographic study.  Aust. N. Z. J. Psychiatry.  34 (3) 484-490. Nicholls, T.L., Petersen, K.L., Pritchard, M.M., & Leech N.M. (2015). Comparing preferences  134  for actuarial vs. structured professional judgment violence risk assessment measures across five continents:  to what extent is practice keeping pace with science? Singh, J.P. (Ed.), APLS Book Series:  International Perspectives on Violence Risk Assessment. New York, US: Oxford Press. Nicholls, T. L. (2001). Assessing insanity acquittees’ risk of violence: Validity of the HCR-20 and PCL:SV for use with women (Unpublished Doctoral Dissertation, Department of Psychology), Simon Fraser University, Burnaby, British Columbia, Canada. Nicholls, T. L., Brink, J., & Greaves, C., Lussier, P., & Verdun-Jones, S. (2009). Forensic psychiatric inpatients and aggression: An exploration of incidence, prevalence, severity, and interventions by patient gender. International Journal of Law and Psychiatry, 32, 23-30. Nicholls, T.L., Brink, J., Desmarais, S.L., Webster, C.D., & Martin, M-L. (2006).  The Short-Term Assessment of Risk and Treatability (START): A prospective validation study in a forensic sample.  Assessment, 13, 313-327. doi: 10.1177/1073191106290559 Nicholls T. L., Gagnon N., Crocker A.G., Brink, J., Desmarais, S., & Webster, C. (2007). START Outcomes Scale (SOS). Vancouver, Canada: British Columbia BC: BC Mental Health & Addiction Services. Nicholls, T., Petersen, K., & Brink, J. (2011, March). Field reliability of START: The relationship between treatment team assessments and diverse patient safety events.  Paper presented at the 4th International Congress on Psychology and Law, Miami, FL. Nicholls, T. L., Brink, J., Desmarais, S. L., Webster, C. D., & Martin, M. (2006). The Short-Term Assessment of Risk and Treatability (START): A Prospective Validation Study in  135  a Forensic Psychiatric Sample. Assessment, 13(3), 313-327. doi:10.1177/1073191106290559 Nicholls, T.L., Brink, J., Desmarais, S.L., Webster, C.D. & Martin, M-L. (2007). The Short-Term Assessment of Risk and Treatability (START): A prospective validation study in a forensic psychiatric sample.  In H. Bloom and C.D. Webster (Eds.), Essential writing in violence risk assessment and management. Toronto: Centre for Addictions and Mental Health. Nicholls, T.L., Brink, J., Desmarais, S.L., Webster, C.D. & Martin, M-L. (2007). The Short-Term Assessment of Risk and Treatability (START): A prospective validation study in a forensic psychiatric sample.  In H. Bloom and C.D. Webster (Eds.), Essential writing in violence risk assessment and management. Toronto: Centre for Addictions and Mental Health. Nicholls T. L., Gagnon N., Crocker A.G., Brink, J., Desmarais, S., & Webster, C. (2007). START Outcomes Scale (SOS). Vancouver, BC Mental Health & Addiction Services. Nicholls, T. L., Webster, C. D., Brink, J., & Martin, M. L. (2008). Short-Term Assessment of Risk and Treatability. In B. Cutler (Eds.), Encyclopedia of psychology and law (pp. 744-746). Thousand Oaks, CA: Sage. Mossman, D. (1994). Assessing predictions of violence: Being accurate about accuracy. Journal Of Consulting And Clinical Psychology, 62(4), 783-792. doi:10.1037/0022-006X.62.4.783 Ontario, Ontario Federation of Community Mental Health and Addiction Programs, 1999.  The Canadian Toolkit for Measuring Psychosocial Rehabilitation Outcomes.  136  Ormston, J. F. (2010). The criminalization of the mentally ill. Canadian Journal Of Community Mental Health, 29(2), 5-10. O’Shea, L. E., & Dickens, G. L. (2014). Short-term assessment of risk and treatability (START): Systematic review and meta-analysis. Psychological Assessment, 26(3), 990-1002. doi:10.1037/a0036794 O'Shea, L.E., Picchioni, M.M., Dickens, G.L. (2014). The predictive validity of the Short-Term Assessment of Risk and Treatability (START) for multiple adverse outcomes in a secure psychiatric inpatient setting. Manuscript submitted for publication. Oshima, I., Mino, Y., & Inomata, Y. (2007).  How many long-stay schizophrenia patients can be discharged in Japan?  Psychiatry and Clinical Neurosciences, 61, 71-77. Perivoliotis D, Granholm E, Patterson T, 2004.  Psychosocial functioning on the Independent Living Skills Survey in older outpatients with schizophrenia.  Schizophr. Res.  69 (2-3) 307-316. Pescosolido B, Wright E, Lutfey K., 1999.  The changing hopes, worries, and community supports of individuals moving from a closing long-term care facility. J. Behav. Health Serv. Res.  26 (3) 276-288. Peternelj-Taylor, C. (2008). Criminalization of the mentally ill. Journal Of Forensic Nursing, 4(4), 185-187. doi:10.1111/j.1939-3938.2008.00031.x Petersen, K., Douglas, K., & Nicholls, T. L. (2011, March). Gender differences in the psychometric properties of the Short-Term Assessment of Risk and Treatability in an acute civil psychiatric sample. Paper presented at the 4th International Congress on Psychology and Law, Miami, FL.  137  Petersen, K., Nicholls, T., Brink, J., Douglas, K., Gagnon, N., Schmitz, N., Leech, N., & Lesage, A. (Submitted).  Forensic and tertiary civil psychiatric inpatients: Comparing psychosocial, clinical, and risk profiles, and the prevalence of negative outcomes.  Psychiatric Services. (Jan., 2015). Petersen, K. L., Nicholls, T. L., Groden, D., Schmitz, N., Stip, E., Goldner, E. M., & ... Lesage, A. (2013). Redevelopment of tertiary psychiatric services in British Columbia: A prospective study of clinical, social, and residential outcomes of former long-stay inpatients. Schizophrenia Research, 149(1-3), 96-103. doi:10.1016/j.schres.2013.05.022 Petersen K. L., Nicholls T. L., Groden D., Schmitz N., Stip E., Goldner E. M., Arnold L. M., Lesage A. D. Redevelopment of tertiary psychiatric services in British Columbia: A prospective study of clinical, social, and residential outcomes of former long-stay inpatients. Schizophrenia research. 2013 Sep;149(1-3):96-103. PubMed PMID: 23815971. Piat M, Lesage A, Bloom D, et al., 2008.  Housing for persons with serious mental illness: consumer and service provider preferences. Psychiatr. Serv.  59 (9) 1011-1017. Potvin S, Briand C, Stip E, et al., 2005.  CANTAB explicit memory is less impaired in addicted schizophrenia patients.  Brain Cogn.  59 (1) 38-42. Priebe S, Turner T., 2003.  Reinstitutionalisation in mental health care.  BMJ.  326 (7382) 175-176. Priebe, S., & Turner, T. (2003). Reinstitutionalisation in mental health care: This largely unnoticed process requires debate and evaluation. BMJ: British Medical Journal, 326(7382), 175-176. doi:10.1136/bmj.326.7382.175  138  Prouteau A, Verdoux H, Stip E, et al., 2004.  The crucial role of sustained attention in community functioning in outpatients with schizophrenia.  Psychiatry Res.  129 (2) 171-177. Reinharz D, Lesage A, Contandriopoulos A., 2000.  Cost-effectiveness analysis of psychiatric deinstitutionalization. Can. J. Psychiatry. 45 (6) 533-538. Royal College of Psychiatrists (2008). Rethinking risk to others in mental health services: Final report of a scoping group. College report CR150. Royal College of Psychiatrists. Salem, L., Crocker, A., Charette, Y., Seto, M.C., Nicholls, T.L. & Côté, G. (2014). Supportive housing and forensic patient outcomes. Law and Human Behavior. Advance online publication. doi:10.1037/lhb0000112 Salvador-Carulla L, Garcia-Mellado M, Velazquez R, Romero C, Alonso F, 1998.  A reliability study of the Spanish version of the social behaviour schedule (SBS) in a population of adults with learning disabilities.  J. Intellect. Disabil. Res. 42 (Pt 1) 22-28. Salvador-Carulla L, Romero C, Johnson S, et al., 2000.  Assessment instruments: standardization of the European Service Mapping Schedule (ESMS) in Spain.  Acta Psychiatr. Scand.  405 (sup) 14-32. Seto, M. C., Harris, G. T., & Rice, M. E. (2004). The criminogenic, clinical, and social problems of forensic and civil psychiatric patients. Law And Human Behavior, 28(5), 577-586. doi:10.1023/B:LAHU.0000046435.98590.55  Sheldon C, Aubry T, Arboleda-Florez J, Wasylenki D, Goering P., 2006.  Social disadvantage, mental illness and predictors of legal involvement.  Int. J. LawPsychiatry. 29 (3) 249-256.  139  Shepherd G., 1995.  The ‘ward-in-a-house’:  residential care for the severely disabled.  Community Ment. Health J. 31 (1) 53-69. Singh, J. P., Grann, M., & Fazel, S. (2011). A comparative study of violence risk assessment tools: A systematic review and meta-regression analysis of 68 studies involving 25,980 participants. Clinical Psychology Review, 31, 499-513. [doi: 10.1016/j.cpr.2010.11.009] Smyth, M., 2006, October 29.  Deinstitutionalization deemed a failure.  The Province, p. A.4. SPSS.  (2001).  The SPSS TwoStep cluster component:  A scalable component enabling more efficient customer segmentation (SPSS White Papers/Technical Report TSCPWP-0101).  Chicago, IL:  Author.  Tarasoff v. Regents of the University of California, 17 Cal. 3d 425, 131 Cal. Rptr.14, 551 P.2d 334 (1976). The Canadian Press. (August 21, 2013). Mental Health Act arrests have quadrupled since 2002: Police chiefs, CTV News. Retrieved from http://bc.ctvnews.ca/mental-health-act-arrests-have-quadrupled-since-2002-police-chiefs-1.1421483 The Standing Senate Committee on Social Affairs, Science and Technology. (2006, May). Out of the Shadows at Last: Transforming Mental Health, Mental Illness and Addiction Services in Canada. Ottawa, ON: Parliament of Canada. Retrieved from: http://www.parl.gc.ca/content/sen/committee/391/soci/rep/rep02may06-e.htm Thomas A, Donnell A, Young T., 2004.  Factor structure and differential validity of the expanded Brief Psychiatric Rating Scale.  Assessment.  11 (2) 177-187. Tiihonen J, Lonnqvist J, Haukka J, et al., 2009.  11-year follow-up of mortality in patients with schizophrenia: a population-based cohort study (FIN11 study). Lancet.  374 (9690) 620-627.  140  Trauer T, Farhall J, Newton R, Cheung P., 2001.  From long-stay psychiatric hospital to Community Care Unit: evaluation at 1 year.  Soc. Psychiatry Psychiatr. Epidemiol.  36 (8) 416-419. Trieman N, Smith H, Kendal R, Leff J., 1998.  The TAPS Project 41: Homes for life? Residential stability five years after hospital discharge.  Community Ment. Health J.  34 (4) 407-417.   Trudel J, Lesage A., 2006.  Care of patients with the most severe and persistent mental illness in an area without a psychiatric hospital.  Psychiatr. Serv.  57 (12) 1765-1770. Velligan D, Prihoda T, Shon S, et al., 2005.  Brief psychiatric rating scale expanded version: How do new items affect factor structure?.  Psychiatry Res.   135 (3) 217-228. Ventura J, Green M, Shaner A, Liberman R., 1993.  Training and quality assurance with the Brief Psychiatric Rating Scale: 'The drift busters.'.  Int. J. Methods Psychiatr. Res.  3 (4) 221-244. Wallace, C.J., Liverman, R.P., Tauber, R., Wallace, J., 2000.  The independent living skills survey: A comprehensive measure of the community functioning of severely and persistently mentally ill individuals. Schizophr. Bull.  26 (3) 631-658. Webster, C. D. (2011). From flipping the coin to seeing both its sides. Journal Of Psychiatric Intensive Care, 7(2), 73-78. doi:10.1017/S174264641100015X Webster, C.D., Martin, M.L., Brink, J., Nicholls, T.L., & Desmarais, S. (2009). Manual for the Short-Term Assessment of Risk and Treatability (START) (Version 1.1). Port Coquitlam, BC: Forensic Psychiatric Services Commission and St. Joseph’s Healthcare. Wilson, C.M, Desmarais, S.L., Nicholls, T.L. & Brink, J. (2010). The role of client strengths in assessments of risk using the Short-Term Assessment of Risk and Treatability  141  (START). International Journal of Forensic Mental Health Services, 9, 282-293. doi: 10.1080/14999013.2010.534694 Wilson, C. M., Desmarais, S. L., Nicholls, T. L., Hart, S. D., & Brink, J. (2013). Predictive validity of dynamic factors: Assessing violence risk in forensic psychiatric inpatients. Law And Human Behavior, 37(6), 377-388. doi:10.1037/lhb0000025 Wilson, C. M., Desmarais, S. L., Nicholls, T. L., & Brink, J. (2010). The role of client strengths in assessments of violence risk using the Short-Term Assessment of Risk and Treatability (START). The International Journal Of Forensic Mental Health, 9(4), 282-293. doi:10.1080/14999013.2010.534694 Vancouver Police Department. (2013). Vancouver’s mental health crisis: An update report. Retrieved from: http://vancouver.ca/police/assets/pdf/reports-policies/mental-health-crisis.pdf  Viljoen, S., Launeanu, M., Hendry, M., Nicholls, T. & Brink, J. (2011, March).Comparing ratings of subject matter experts and clinicians on the START in a forensic clinic. Poster presented at the 4th International Congress on Psychology and Law, Miami, FL. Viljoen, S., Nicholls, T.L., Greaves, C., de Ruiter, C., & Brink, J. (2011) Resilience and successful community reintegration among female forensic psychiatric patients: A preliminary investigation. Behavioral Sciences and the Law, 29(5), 752-770. doi: 10.1002/bsl.1001 de Vogel, V., de Ruiter, C., Bouman, Y., & de Vries Robbé, M. (2009). SAPROF: Guidelines for the assessment of protective factors for violence risk [English version of the Dutch original]. Utrecht, The Netherlands: Forum Educatief.  142  Webster, C. D., Martin, M. L., Brink, J., Nicholls, T. L., & Desmarais, S. (2009). Manual for the Short-Term Assessment of Risk and Treatability (START) (Version 1.1). Port Coquitlam, BC: Forensic Psychiatric Services Commission and St. Joseph’s Healthcare. Webster, C. D., Martin, M. L., Brink, J., Nicholls, T. L., & Middleton, C. (2004). Manual for the Short-Term Assessment of Risk and Treatability (START) (Version 1.0 Consultation Edition). Port Coquitlam, BC: Forensic Psychiatric Services Commission and St. Joseph’s Healthcare. Webster, C. D., Douglas, K. S., Eaves, D., & Hart, S. D. (1997). HCR-20: Assessing risk for violence (Version 2). Vancouver, Canada: Mental Health, Law, & Policy Institute, Simon Fraser University. Webster, C.D., Martin, M.L., Brink, J., Nicholls, T.L., & Desmarais, S. (2009). Manual for the Short-Term Assessment of Risk and Treatability (START) (Version 1.1). Port Coquitlam, BC: Forensic Psychiatric Services Commission and St. Joseph’s Healthcare. Webster, C. D., Nicholls, T. L., Martin, M. L., Desmarais, S. L., & Brink, J. (2006). Short-Term Assessment of Risk and Treatability (START): The case for a new violence risk structured professional judgment scheme. Behavioral Sciences and the Law, 24, 747-766. Webster, C.D., Nicholls, T.L., Martin, M-L., Desmarais, S.L., & Brink, J. (2006). Short-Term Assessment of Risk and Treatability (START): The case for a new structured professional judgment scheme.  Behavioural Sciences and the Law, 24, 747-766. doi: 10.1002/bsl.737 Webster, C. D., Martin, M. L., Brink, J., Nicholls, T. L., & Middleton, C. (2004). Manual for the Short-Term Assessment of Risk and Treatability (START) (Version 1.0 Consultation  143  Edition). Port Coquitlam, BC: Forensic Psychiatric Services Commission and St. Joseph’s Healthcare. World Health Organization. (2001). Mental Health:  New Understanding, New Hope.  World Health Report 1020-3311. Geneva:  WHO. Wilson, C., Desmarais, S., Nicholls, T.L. & Brink, J. (2010). The role of strengths in assessments of short-term violence risk. International Journal of Forensic Mental Health, 9, 282-293. Wilson, C. M., Desmarais, S. L., Nicholls, T. L., Hart, S. D., & Brink, J. (2010). Incremental validity of dynamic factors in the assessment of violence risk using the HCR-20 and START. Manuscript submitted for publication. World Health Organization, 2003.  Mental Health Policy and Service Guidance Package:  Organization of Services for Mental Health.   Wykes T, Sturt E., 1986.  The measurement of social behaviour in psychiatric patients: An assessment of the reliability and validity of the SBS schedule.  Br. J. Psychiatry.  148 1-11. Wykes T, Wing J., 1982.  A ward in a house: accommodation for "new' long-stay patients.  Acta Psychiatr. Scand.  65 (5) 315-330. Yudofsky, S. C., Silver, J. M., Jackson, W., Endicott, J., & Williams, D. (1986). The Overt Aggression Scale for the objective rating of verbal and physical aggression. The American Journal Of Psychiatry, 143(1), 35-39. Yudofsky, S. C., Silver, J. M., Jackson, W., Endicott, J., & Williams, D. (1986). The Overt Aggression Scale for the objective rating of verbal and physical aggression. The American Journal Of Psychiatry, 143(1), 35-39.  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0166451/manifest

Comment

Related Items