UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A comparison of self-reported use of primary mental health care versus provincial administrative health… Palin, JoAnne 2008

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
24-ubc_2009_spring_palin_joanne_.PDF [ 3.38MB ]
Metadata
JSON: 24-1.0071438.json
JSON-LD: 24-1.0071438-ld.json
RDF/XML (Pretty): 24-1.0071438-rdf.xml
RDF/JSON: 24-1.0071438-rdf.json
Turtle: 24-1.0071438-turtle.txt
N-Triples: 24-1.0071438-rdf-ntriples.txt
Original Record: 24-1.0071438-source.json
Full Text
24-1.0071438-fulltext.txt
Citation
24-1.0071438.ris

Full Text

A COMPARISON OF SELF-REPORTED USE OF PRIMARY MENTAL HEALTH CARE VERSUS PROVINCIAL ADMINISTRATIVE HEALTH RECORDS  by JoAnne Palm  A THESIS SUBMIfl’ED IN PARTIAL FULFILLMENT OF ThE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY In  THE FACULTY OF GRADUATE STUDIES (Health Care and Epidemiology)  THE UNIVERSITY OF BRITISH COLUMBIA (Vancouver) October 2008  C JoAnne Palm, 2008  ABSTRACT  Most mental health (MH) care in Canada is provided by General Practitioners (GPs). Thus. information about use of primary MI-I care is fundamentally important to  researchers and policy makers. The two predominant sources of data on primary MN care are self-reported data, and provincial administrative databases which capture physician reimbursement data for medical services provided to virtually all residents, under Canada’s publicly-funded universal health care system. The objective of this research was to compare estimates of primary MI-I care from the two types of data over a 12 month reference period, using an individual-level data linkage in a sample of 2,37X residents of the province of British Columbia. The proportion of individuals in this sample who had MI-I care from a GP was approximately twice as high in the administrative data (19.3%) versus the self-reported data (8.5%). in contrast, the number o/ A4H visits to each individual’s main GP tended to be higher in the self-reported data than the administrative data. In terms of agreement. three-quarters of individuals who had primary MH care according to the administrative data did not report such care. while one-third of individuals who reported primary Mu care did not have an administrative record of such care. The study also explored whether individuals in the sample who had had a major depressive episode (MDE) in the 12 month reference period, were more or less likely than individuals who had not had an MDE to have had primary MI-I care according to both data sources versus only one data source. The results of the study indicated that individuals who had hat/an A/IDE had a greater likelihood of having had care according to both data sources. The self-reported data were from the 2002 Canadian Community I-Iealth Survey on Mental Health and Well-Being, and the method used to identify primary MI-I care in the administrative data closely paralleled the method that has recently been proposed for national surveillance ofMH in Canada. It is hoped that the findings will help researchers. policymakers and methodologists to make informed choices when collecting. analyzing and interpreting each type of data.  II  TABLE OF CONTENTS  ABSTRACT  .  TABLE OF CONTENTS  .  ii iii  LIST OF TABLES  vi  LIST OF FIG(JRES  vi  ACKNOWLEDGMENTS DEDICATION CO-AUTI IORS HIP STATEM ENT  vii  viii ix  I INTRODUCTION I 1.1 OVERVIEW OF TI-IE DISSERTATION 1,2 BACKGROUND AND RATIONALE 4 1.3 LITERATURE REVIEW 7 1.3.1 Comparisons of self-reported primary health care versus administrative records 9 1 .3.2 Comparisons of self-reported MI-i care versus provincial administrative records 11 1.3.3 Specific mental disorder and agreement between self-reported ambulatory MIl care versus provincial administrative data 15 1.3.4 Methodological gaps in agreement studies 17 1.4 A BROADER RESEARCH FRAMEWORK 19 1.5 AiMS AND OVERVIEW OF THE PRESENT STUDY 22 1.6 REFERENCES 26 2 USE OF GENERAL PRACTITIONER SERVICES FOR MENTAL HEALTH ISSUES: AGREEMENT BETWEEN SELF-REPORTED DATA AN[) ADMINISTRATIVE DATA 2.1 INTRODUCTION 2.2 METHODS 2.2.1 Data sources 2.2.1.1 Self-reported data 2.2.1.2 Administrative data 2.2.1.3 Data linkage and study sample 2.2.2 Definitions: Primary MR care 2.2.3 Analytic techniques 2.3 RESULTS 2.4 [)ISCUSS ION  32 32 37 37 37 37 38 39 41 41 43  Ill  54 2.5 CONCLUSION 56 2.6 REFERENCES 3 AGREEMENT BETWEEN ES FIMA1 ES OF PRIMARY MENl Al HEAl III ( ARI IN SELF-REPORTED I)ATA VERSUS AI)MINISTRATIVE DATA AMONG 59 DEPRESSED AND NON-DEPRESSED INDIVIDUALS 66 3.1 METHODS 66 3.1.1 Data sources 66 3.1.2 Data linkage and sample size 67 3.1.3 Key independent variable: Major Depressive Episode (MDE) 69 3.1.5 Dependent variables (categories of agreement) and statistical analyses 70 3.2 RESULTS 71 3.2.1 Negative Agreement versus Disagreement 3.2.2 Positive Agreement versus Disagreement 72 3.2.3 Disagreement (Adminisiralive record only) versus Disagreement (e/frepori only) 72 3.3 DISCUSSION 75 3.4 CONCLUSION 81 3.5 REFERENCES 82 .  4 NUMBER OF VISITS TO GENERAL PRACTITIONERS FOR MENTAL HEALTH CARE IN A 12 MONTH REFERENCE PFRIOD A COMPARISON BI TWI I N SI LI -RFPORTED DAI A AND ADMINISTRATIVE DATA 85 4.1 METHODS 8$ 4.1.1 Data sources 88 4.1 .2 Data linkage and sample size 89 4.1.3 Definition of a primary MH care visit self-reported data 90 4.1 .4 Definition of a primary MH care visit administrative data 91 4.1.5 Statistical analyses 91 4.2 RESULTS 93 4.3 DISCUSSION 97 4.4 CONCLUSIONS 1 04 4.5 REFERENCES 106 —  -  5 CONCLUSION 5.1 SUMMARY OF RESEARCH FINDINGS 5,2 IMPLICATIONS AND GUIDANCE FOR RESEARCHERS WHO USE PRIMARY MENTAL HEALTH CARE DATA 5.3 CONTRIBUTIONS 5.4 FUTURE RESEARCH 5.5 CONCLUSIONS 5.6 REFERENCES  113 116 118 121 123  Appendix A  125  Appendix B  127  109 109  iv  Appendix C. 129  V  LIST OF TABLES Table 1.1: Summary of three comparisons between self-reported health care and provincial administrative health records  15  Table 2.1: Mental health care from a general practitioner in the past 12 months according to seli 43 reported data and administrative data Table 3.1: Adjusted and unadjusted associations between depression and agreement categories, 73 with Negative Agreement as the reference category Table 3.2: Adjusted and unadjusted associations between depression and agreement categories, 74 with Positive Agreement as the reference category Table 3.3: Adjusted and unadjusted associations between depression and agreement categories, with Disagreement (Admin record only) as the reference category 74 1’abIe 4.1: Number of visits to General Practitioners in the past 12 months in self-reported data and administrative data 95 Table 4.2: Direction of difference in the number of mental health visits to respondents’ main General Practitioners in the past 12 months in the self-reported data versus provincial administrative health records (MSP) 96 Table A.1: Distributions of demographic and mental health status indicators in the complete CCHS Public Use Microdata File for British Columbia, and the linked study sample 126  vi  LIST OF FIGURES 3  Figure 1.1: Illustration of the research focus Figure 1 .2: Factors associated with MH care use  16  Figure 1.3: The present study within a broader research context  20  Figure 3.1: Labels for each category of agreement  63  Figure 3.2: Logistic regression analyses to assess whether depressed individuals were more likely than non-depressed individuals to be classified in the Agreement groups or I)isagreernent groups for primary MI-I care use the past 12 months 65 Figure 3.3: Number of individuals in each agreement category  71  Figure 4.1: Number of individuals in the study sample, and in the study sub-sample fbr the principal analyses  92  vi  ACKNOWLEI)GMFNTS  Dr. Clyde i—Tertzman’s support as my research supervisor made this work possible. He introduced me to epidemiology, and his insights have formed the basis of my scholarly education. Dr. Mieke Koehoorn shared her abundant knowledge of epidemiological principles as my co-supervisor, and strengthened the content of this thesis with her meticulous attention to detail. Dr. Elliot Goldner provided invaluable clinical and policy perspectives, in addition, he and Dr. Sam Sheps supplied helpful advice and opportunities as my mentors in training programs in health services and mental health policy research. Dr. Bruno Zumbo supported a successful grant application for this research. Dr. Hubert Wong, I)r. Kim McGrail, Wayne .lones, i)r. Elizabeth Lin and Dr. Scott Patten generously shared their expertise. Lisa Chen patiently provided SPSS statistical software programming support and advice during the ftundational stages of creating the analytic database for this research. I also owe thanks to Dr. Michael Wolfson and his expert stalTat Statistics Canada as well as the many helpftil staff at the Centre for l-lealth Services and Policy Research (Cl ISPR) and the British Columbia Ministry of Health; and I offer special thanks to the respondents to the Canadian Community Health Survey on Mental Heath and Well-Being (CCIIS). I also extend thanks to numerous valued colleagues and friends in the Department of Health Care and Epidemiology. In particular, I thank Dr. Gary Poole, who sparked my interest in health research and  has supported my career in many ways over the years. I especially thank my family lir  their tremendous and unwavering support, generosity, encouragement, advice, and assistance. This research was supported by awards from the Canadian Institutes for health Research (CIHR); the CIHR Strategic Training Program for Research in Addictions and the Mental lIealth Policy and Services; the Western Region Training Program for Health Services Research (funded by CIHR, the Alberta Heritage Foundation for Medical Research. and the Canadian Health Services Research Foundation (Cl-ISRF)); and alointly-funded award from Cl-ISRF and the British Columbia Health Research Foundation (predecessor of the Michael Smith Foundation for I-Iealth Research). Additional scholarship and bursary support was received from Mensa Canada, the University of British Columbia, and the Department of Health Care and Epidemiology. Grant ftinding was received from CIT-JR (with JoAnne Palm as Collaborator, and Bruno Zumbo as Principal Investigatory. The British Columbia Ministry of Health linked the CC1IS data (tiom Statistics Canada) to provincial administrative data. and CHSPR extracted the administrative data required for this thesis.  vii  DEDICATION  In memory of Dr. Alexander Leighton (1908 -2007) who pioneered the application of epidemiological methods to the study of mental  conditions  and who inspired and nurtured generations of scientific researchers.  “1-Ic had the vision to become a pioneer, the questioning mind and the intellectual skills required to find satisfactory answers to multifarious phenomena, the health and stamina to persevere and achieve positive results in everything he undertook. and the type of charismatic personality to attract close collaborators from many different fields.” Marc-A dé/ard TrembJay Transculiural Psychiairy, Vol. 43, No. 1, 7-20 (2006)  viii  CO-AUTHORSHIP STATEMENT  This statement describes the contribution of each of the following individuals to the manuscripts in this dissertation, in accordance with the requirements of the Faculty olGraduate Studies at the University of British Columbia. JoAnne Palm initiated and developed the research questions for this dissertation. She applied for a data linkage between the British Columbia Linked Health Database and the Canadian Community Health Survey, cycle 1.2. She formulated the methodology, oversaw initial data checking, prepared the data, and conducted all the analyses. She interpreted the results and authored all the manuscripts. She also authored a successful grant application to the Canadian Institutes for Health Research (CIHR) to support these analyses, and Dr. Bruno Zumbo was the Principal Investigator on the grant. Dr. Clyde Hertzman was the thesis co-supervisor. He provided ongoing guidance on the thesis research and methodology, and the presentation of findings. 1-Ic reviewed all the manLlscripts and provided feedback. I-Ic was a co-investigator on the above-mentioned CIt 1k grant. Dr. Mieke Koehoorn was the thesis co-supervisor. She provided on-going guidance on the thesis research, including the scope of the research, and the interpretation of findings. She reviewed all manuscripts and provided feedback. Dr. Elliot Goldner was a member of the thesis committee. He provided guidance, reviewed the manuscripts and provided feedback on aspects of the thesis which pertained to mental health practice and policy. I-Ic was a co-investigator on the above-mentioned CIlIR grant. Dr. Bruno Zumbo, was a member of the thesis committee, and advised on specific methodological issues, particularly those pertaining to measures of agreement. He reviewed the draft manuscripts and provided guidance on the structure of the manuscripts. As noted, I)r. Bruno Zumbo was the Principal Investigator on the above-mentioned CIHR grant.  ix  I  INTRODUCTION  1.1 OVERVIEW OF THE DISSERTATION  Mental disorders have been ranked second only to cardiovascular diseases in terms of direct health care expenditures in Canada, and they are counted among the top 10 disease categories in terms of overall economic burden to Canadians (including lost productivity)L. By 2020, depression is expected to he the second leading cause of disease burden worldwide, after ischaemic heart disease . In addition to monetary costs, the symptoms and stigma associated 2 with mental disorders can exact staggering personal and social burdens on individuals and their lamilies 3 Access to appropriate mental health (MH) care, and the appropriate allocation of limited Mu care resources, form integral parts of the policy arsenal aimed at alleviating the societal and economic costs of mental disorders. Individuals obtain MH care from an array of sources, including psychologists, counsellors, social workers, nurses, chaplains, alternative health care providers, support groups, and physicians. Of these, General Practitioners (GPs) provide most ambulatory MH cal-c in 4 Canada ’ . Four out of five Canadians see a GP at least once in a given . and so the vital role of GPs in the detection, prevention and treatment of Mil issues is the 6 year focus of considerable research and policy attention 7  8  Thus, accurate  information on  the  use  of  MIl care in general practice settings is fundamentally important to health researchers and policymakers, in order to support evidence-based policy decisions and treatment guidelines. Basic information needs include: I.  The number of individuals who receive j’t’lff care from GPs This information is the basic building block of informed policy decisions aimed at assessing and improving the allocation of MI-I resources: and for developing and evaluating service  delivery models designed to improve access to cat-c and the effectiveness of treatment. 2. 11w characteristics 0/individuals who receive Iv!H care from UPs and those who do not This information is necessary in order to investigate and optimize the alignment of needs with care. Depending on the research or policy question, the characteristics of interest may  include demographic traits, environmental factors, or specific MR issues.  3. The  iniensily of MH  cure individuals  recen’e/ronl  (iFs  (measured by the nunther  0/ viSils)  This information is used to estimate the direct health care costs of mental disorders: to quantify the intensity of service use among individuals with mental disorders versus those without mental disorders and to evaluate and improve the adequacy of MH treatment frllow up.  Since it is not practical for researchers or policy analysts to directly observe large numbers of medical encounters in order to gather this information for population-level studies, they generally must depend on indirect sources of information. In Canada. the two predominant sources of population-level data on MH care are:  1. Self-reports, which are based on the recollection and reporting of MI-I care by survey respondents; and  2. Provincial administrative health care records, which are comprised of physician reimbursement records for medical services provided to virtually all provincial residents.,  under Canada’s universal health care system.  These two data sources have largely informed current knowledge about the use, and users. of MI--I care in Canada. Yet, there is growing evidence suggesting that the two data sources may  yield identical pictures of MFI care ’ 9  fbi  Studies that have been based on provincial  administrative health records have appeared to produce higher estimates of the prevalence ol primary MI—I care use than studies based on self-reports ’’. This may suggest that Ml! research 9 findings could he sensitive to the data source used to measure MH care. If researchers and policvmakers use these two data sources interchangeably. without awareness ol the differences between them, this could potentially result in flawed conclusions; missed opportunities ibr beneficial interventions; or unexpected or unintended outcomes. Figure 1.1 illustrates the research focus for this dissertation. The objective of this work is to directly compare self-reported use of GP services for MR care versus administrative records on a person-by-person basis in a sample of 2,378 residents of the province of British Columbia. i’he methodology involves analyses of existing survey data and secondary analyses of computerized  provincial health insurance records that were linked at the individual-level, and that were transformed into a researchable database for the investigations described in this  dissertation.  Figure 1.1: Illustration of the research focus  Do these two data sources yield similar estimates?  DATA based on self-reported MH care (from patients)  ‘s  USE OF MENTAL HEALTH CARE  DATA based on provincial administrative records of MH care (from physician billing records) I , I, I, I,  The aim of this research is to illuminate possible differences between self-reported data and  provincial administrative records in order to strengthen the evidence base of MI-I services and policy research, and ultimately the MH care of Canadians. This dissertation is intended to provide evidence for consideration by survey methodologists, administrative data analysts, researchers, and government agencies, to help them to:  •  Make informed choices about the most appropriate data source for a specific research question. and the potential impact of a particular data source on research findings:  •  Consider if, and how. the perspectives provided by each data source can be appropriatel’ optimized by linking the two data sources:  •  Identify priorities for research to understand reasons for differences between the two data sources, and to ameliorate them (if necessary).  The structure of the dissertation follows the manuscript-based format prescribed by the Faculty of Graduate Studies at the University of British Columbia. It consists of five chapters. The  3  present chapter will describe the background and rationale for the study, and the place of this research in the context of the recent evolution of data collection for MH services research. Next, it will provide a summary of the literature. Then, it will conclude with a description oIthe research questions. Chapters 2 through 4 are individual manuscripts:  •  Chapter 2 will describe the proportion of individuals in the study sample who used GP services for MH issues in the past 12 months according to self-reports versus administrative data. In addition, it will examine agreement between self-reported MH care versus each individual’s provincial administrative records.  •  Chapter 3 will describe whether agreement is randomly distributed, or if individuals who had a specific and common mental disorder, such as depression, were more likely than individuals who did not have that disorder to have had MH care from a UP according to both data sources, or one data source hut not the other, or neither data source.  •  Chapter 4 will examine whether or not the number of self-reported MH visits to each individual’s main UP tended to be higher, lower, or equal to the corresponding number of visits in the administrative data.  The dissertation will conclude with Chapter 5, which will summarize the findings and unique  contributions of the dissertation, and will outline key implications and recommendations for researchers, policyrnakers, survey designers and administrative data analysts.  1.2 BACKGROUND AND RATIONALE  Self-reported information underlies an overwhelming portion of research on human health and behaviour. In health research and clinical practice, self-reported information is generally collected when:  •  A subjective assessment is sought, such as self-perceptions of health status; or  •  A respondent is the only source of information about concrete events or behaviours; or  •  Alternative sources of information may be available, but the collection of self-reports is the most convenient or efficient method for obtaining data’ . 2 4  The topic of this dissertation concerns the latter case. Surveys have traditionally provided a more efficient and feasible method of collecting population-level MI-i care data than medical chart reviews. However, in an era in which data is rapidly becoming digitized. computerized administrative records increasingly present an alternative to self-reported data about health care.  In order to provide context for each type of data source in the field of MI-I services and policy research, it is useful to briefly consider the recent evolution of MR data collection. In recent years, three notable MI—I surveys in Canada have provided valuable information to inform MI-I research. hut due to their limited geographic coverage they did not provide national perspectives  .  Very little detailed information on mental disorders and MI-I services was  available from a national survey in Canada until recently. Most self-reported information on mental illness and MH care was gathered through a small number of questions in an on-going national survey of general health (the National Population Health Survey (NPHS)), and a newer cross-sectional survey (the Canadian Community Health Survey (CCHS)) that was launched in 2000/01.  The CCHS is an annual national survey, and every second year it focuses on a specific health issue. In 2002, it focussed on mental health. The 2002 Canadian Community Health Survey on Mental Health and Well-being (CCI-IS, cycle 1 .2) was launched in collaboration with the World Mental Health Survey Initiative which involved contemporaneous surveys in more than 20 . As Canada’s first national MR survey, it provides detailed information about the use 16 countries of MH services and correlates of care . t7  Enthusiasm about the potential riches of survey data has been tempered with concerns about the ability of respondents to accurately recall and report the occurrence of MR care. The reliance on unverified self-reports continues to be cited frequently as a limitation in studies based on survey ’ 5 data  1821  Apart from concerns about the limitations of self-reports, it is recognized that  national MR surveys will be conducted relatively rarely in Canada, so researchers and policymakers also desire MR care data that can provide on-going and longitudinal perspecivcs in 2006, the Canadian Standing Senate Committee on Social Affairs, Science and ‘I’cchnology,” chaired by Senator Kirby, released a report pertaining to the need to transform mental health and  addiction services in Canada . The Committee remarked on the dearth of information available 22 to evaluate the quantity and quality of current MR services and programs. Their report, commonly known as the Kirby Report, commented on the needs expressed by MN advocates for a national surveillance system that could be used for the planning, implementation and evaluation of MH services, policies and programs; and that could augment data from surveys and other sources. Recently, the Public }-Iealth Agency of Canada (PHAC) began assessing the feasibility of using provincial administrative health care data for MR surveillance purposes 1  In Canada, provincial governments maintain administrative databases of physician reimbursement records. As noted previously, these databases cover health care provided to virtually all provincial residents because Canadians receive publicly-provided universal health insurance. An administrative record for ambulatory care generally includes a date of service; a description of the service provided by a physician to a patient; and the reason for a medical visit, commonly in the form of a diagnostic code from the World Health Organization’s International Classification of Diseases . The diagnostic codes are assigned by physicians or medical office 23 staff when submitting billing records. PHAC plans to use these diagnostic codes to identify mental disorders in administrative health care records. While administrative data have already formed the basis of several scholarly publications and government reports on the use of care Ibr MR , 24 PHAC’s feasibility report is expected to usher in more widespread use of this 9 ’ 7 issues methodology for MH research, policy, and planning purposes.  As a result of the existence of these two sources of data, there is one body of research that has used administrative data to study MR 7 care 9,24, while other published analyses have used survey ’ data to study MR care ’ 10  1925  Recent agendas of the annual scientific symposia of the Canadian  Academy of Psychiatric Epidemiology (CAPE) illustrate this point. The members of CAPl.i include researchers from virtually all major academic research centres for psychiatric epidemiology in Canada, and they have worked collaboratively with Statistics Canada on (‘CHS content, and with the Public Health Agency of Canada on its national surveillance initiative. Following the release of the CCHS, a CAPE symposium, as well as a special edition of the Canadian Journal of Psychiatry, were dedicated to research based on CCHS data . More 26 recently, the 2007 CAPE scientific symposium focussed on studies and ideas pertaining to the use of provincial administrative data for research and policymaking. The dialogue on how the Iwo data sources compare is in its nascent stages. 6  The demand to understand differences between the two data sources has become more exigent in  the wake of the surveillance recommendations of the Kirby Report and the announcement of plans for a second CCI-lS on mental health. However, the existing literature does not provide sufficient information to assess the comparability of the predominant data sources for MI-I services research in Canada.  1.3 LITERATURE REVIEW  An epistemological tension can arise when comparing two representations of a similar phenomenon, because data from administrative databases tend to he viewed as more ohiecti ye. less subject to failures of recall. and therefore more “valid” than seZ reported information. lo address the contention that administrative data are more valid than self-reported dala, it is important to make a distinction between administrative records and medical records. Administrative records are designed for reimbursement purposes, not to track detailed medical hi stories.  Within fee-for-service reimbursement systems, in which physicians must submit claims for each service in order to be reimbursed, there are financial incentives by both the payee and Payer to ensure that billing records reflect the amount of services used. However, in order to identify visits to GPs for MH issues, PHAC will use diagnostic codes. This is also the only method for identifying primary MN care in the administrative data in British Columbia, the province in which the present study was conducted. The diagnostic codes in British Columbia’s physician reimbursement records are rarely audited and do not guide a patient’s treatment, so there is limited incentive for a physician to ensure the accuracy and precision of each and every diagnostic code that is submitted with billing records. In British Columbia, a physician may only submit one diagnostic code with each billing record to describe the reason for the service. 1-lowever, if physical and mental health issues were discussed, a single diagnostic code may not reflect the full content of a visit, and so some primary MR care may be overlooked if the MN diagnosis were not recorded on the hilling record. Conversely, a GP may initially suspect that a patient has a ME-I issue, and record a MR diagnosis code. If the ME-I issue were later ruled out, the visit would still remain in the administrative database and the individual may be ‘rnisclassi1Ied’ as primary MH care user. 7  Some GP visits are not captured in the province’s administrative database. Although GPs in British Columbia are primarily paid on a fee-for-service basis, a portion of medical services in the province are provided under alternate payment arrangements, which can include salaries or sessional payments. In addition, some information about health care received outside the province/country may not appear in the provincial administrative database.  In light of these issues, the paradigm underlying the present research is that either data source  (self-reports or administrative records) is potentially fallible, so neither one is considered upriori to he an objective representation of reality, and therefore neither is considered to be a “gold standard” measure of the use of primary MH care. The aim of this research is to explore agreement between two common sources of data on the use of GP services for MI-I care, in order to illuminate to what extent they may yield similar or disparate tindings. It is not a validation study of either data source.  While many agreement studies in the literature acknowledge that neither data source is considered to be a gold standard, the terms “under-reporting” and “over-reporting” are frequently used to describe the relationship between self-reports and alternate sources of data. However, this language suggests that the discrepancies between the two data sources are due to errors in the self-reports. Therefore, in the present study, the terms “low reporting” and “high reporting” are used to describe the direction of difference between the sources of data. For example. “high reporting” simply refers to a self-reported estimate that is higher than the alternate source of information, but the terminology is not meant to imply that the reason for the discrepancy should necessarily be attributed to “over” reporting by a respondent. “High reporting” could result from an underestimation of service use in the administrative data, or an overestimation of service use in the sel[reported data.  On another note about terminology: throughout this dissertation a medical service that is provided by a GP is referred to as “primary care” for the sake of brevity. Although the term “primary care” can encompass a wider range of health care providers, it refers only to UP services in this document.  8  The literature that is summarized in the next sections conForms as closely as possible to the design and objectives of the present research by focussing on comparisons between sell-reported use of ambulatory care and provincial administrative health care data, in a community sample. This literature review addresses four areas. The first pertains to comparisons between selfreported use of primary care versus provincial administrative records. The second focuses on comparisons that pertain specifically to the use of care for MHissues. The third pertains to the association between having had a specific mental disorder during the 12 month retèrence period and having agreement between the two data sources. The fourth pertains to methodological issues in agreement studies.  1.3.1 Comparisons of self-reported primary health care versus administrative records  It is useful to examine the results of agreement studies pertaining to the use of GP services For any health issue before focussing on studies of the use of GP services for MI-f care specifically.  Mustard ci al (n  =  8,529) compared data from the 1994/95 National Population I-lealIh Survey  (NPHS) with administrative data from four Canadian provinces, using individual-level linkages, and found that the proportion of individuals in the sample who used GP services was approximately 80% in both data sources . 6  Raina eta/conducted a comparison of administrative data from the province of Ontario and selfreported data that was collected using questions that were based on NPHS questions in a community-based sample of seniors, age 65 and older (n  =  1 ,038) 27 In that study, the proportion  of seniors (91.4%) who reported using primary care exceeded the proportion that appeared in the administrative data (79.3%). By comparison, Mustard ci al, found that the proportion ol primary care users in the self-reported data and the administrative data in the 65 and older age group were equal (approximately 89% in both data sources). The latter estimates were based on data From all thur provinces, and not only Ontario, so perhaps this contributed to the different findings in the two studies. It is useful to note that Raina et al reported that the apparent high reporting of primary care did not appear to be due to misclassification of specialists as GPs. In terms of individual-level agreement, Raina ci al reported that the proportion of observed agreement between self-reports and administrative data was 79.3%. However, when beyond chance agreement was assessed using the Kappa statistic, agreement was poor to thir (0.1 95). ()  When the prevalence of a phenomenon is particularly low or high. as noted by the authors. observed agreement can provide a more appropriate assessment of overall agreement than . Mustard 27 Kappa  ci  al did not report observed agreement, but they reported a moderate level of  agreement based on Kappa (0.51).  In terms of the number of visits, Mustard  ci  al reported that the average number 0/visits to GPs  differed little in the administrative records (5.17) versus the self-reported data (4.79) ftom the fbur provinces. For the sample from British Columbia, the discrepancy was larger  the average  number of visits was slightly higher (14.5%) in the administrative data (5.90) than the selfreported data (5.06). Raina ci a! did not report their findings in the same format, hut their results demonstrated a tendency toward low reporting of visits: 59.4% of seniors were lo reporters (most of whom had at least 3 more visits in their administrative records than they reported). 29.6% were high reporters, and only 13.6% had exact agreement (see Table 1 .1).  Other studies are less applicable to the current study. In the existing literature, there are substantial variations in the composition of study populations, health care systems, sampling methods, duration of recall periods, types of health care providers or settings, levels of data aggregation. methods for collecting self-reports, sources of alternate data, and methods for assessing agreement and reporting results . For example. Ungar ci a! compared provincial 28 administrative records in Ontario to self-reported health service use among individuals enrolled in a project pertaining to the use of bronchial inhalers  (ii  =  83). Their study fbcusscd on visits  with diagnostic codes pertaining to respiratory issues; it included general internists in the definition of GPs: and it involved multiple interviews with respondents. Thus, their findings  must he generalized to the present study with caution. The proportion of users of GP care were equal in the baseline interview, but a tendency toward low reporting was observed in the subsequent interviews. Exact agreement for the nmnher of visits occulTed in 65% of cases hut direction and magnitude of discrepancies were not reported.  Ritter c/ a! compared computerized utilization records from an American Health Management Organization (1-IMO) with self-reports of 216 volunteers, and found an average diffrence of I .06/wer self-reported physician visits than the number of visits found in the computerized records. Exact agreement for the number of visits occurred in 22% of cases . Visits to all types 3 of physicians were included (except psychiatrists), and the sample was not representative of the  10  general population: they were volunteers, age 40 and older, who had enrolled in a chronic disease self-management course.  While there appears to be a general tendency toward low reporting of the number of GP visits, a very commonly-cited agreement study by Cleary and Jette (1984) reported that, on average, individuals reported 0.5 more visits than were found in their medical records . Thai refirs to the 31 net difference, which includes the offsetting effects of discrepancies in opposing directions. As noted by Glandon ci al, calculations based on the data presented in Cleary and .Jeite’s study indicate that the absolute difference between the two data sources (regardless of the direction of difference) was larger (1 .8 visits) than the net 3 difference 32 The study included a representative sample of 908 individuals in Wisconsin, aged 18 and older, and was based on a 12 month recall period like the present study. No distinction was made between GPs and other types of physicians. This study was included in this literature review because it was a landmark study that has been frequently cited, however it is important to recognize that it was based on medical recordv rather than administrative data.  Main observations  •  Of the studies summarized in the previous section, those by Mustard ci a! and Raina ci a! are most applicable to the present study because they compared the self-reported use of GP services versus prov!ncia! administrative data. However, the generalizahility of the findings by Raina and colleagues to the present study are somewhat limited by the focus on individuals 65 and older.  •  Based on the available information, it appears that the proportion of individuals who use GP services may be similar in both self-reports and provincial administrative data, although Raina ci al (but not Mustard et al) observed high reporting among seniors. The number of visits tend to be lower in self-reports versus administrative records.  1.3.2 Comparisons of self-reported MH care versus provincial administrative records  Only one study was found that directly examined agreement between self-reported MJJ care and provincial administrative records, at the individual level. Rhodes et al compared self-reported 11  data from the 1994/95 NPI-IS with administrative data in the province of Ontario  (11  . 3 4621Y  The 1994/95 NPHS questionnaire asked if respondents had talked to a health professional about their emotional or mental health. and did not refer exclusively to GPs. The authors  compared  the  sell-reported data to physician hilling records in the administrative data (hut not specifically OP records). Both data sources (self-reports and administrative data) yielded the same proportion of MH care users (approximately 9% of the study sample). Approximately 5% of the individuals in the sample were high reporters; approximately 5% were low reporters; 4% had MI-I care according to both data sources; and 86% had no MH care according to either data source. Owing to the large size of the latter category, the overall percentage agreement was high (90.2%). The authors reported that the mean number of visits was 28% higher in the self-reported data than the administrative data (see Table 1.1).  Although the study by Rhodes ci a! compared self-reported MH care to provincial administrative data, the focus of their study differed in a fundamental way from the focus of the present study. The present study uses diagnostic codes to identify MH care in provincial administrative records, The study by Rhodes ci a! focussed on a different methodology for identifying Mil care in administrative records. In Ontario, GPs may address MH issues during general medical visits (as they do in British Columbia); or they may bill the government for MI-I-specific services, like psychotherapy (which they cannot do in British Columbia). As a result, there are two methods for identifying MR care in Ontario’s administrative data. MH care provided during genera! office visits can be identified using diagnostic codes for MH conditions. MI—I-specific services can be identified using service codes, and records with such service codes usually include a MI-I diagnosis as the reason for the visit. Thus, when identifying MI-I care in Ontario’s administrative data, diagnostic codes capture a substantial number of MR care users who are not captured by using MI—I  service  codes alone 9  3436  Rhodes ci a! used only service codes to identil’ Mll care in  the administrative data, and so individuals who only received MR care during genera! medical visits were classified as  non-users  of MH care services for the purposes of their study. It has  been estimated that MI-I service codes alone may capture half as many MR care users in Ontario as the diagnostic code method , and so the results of the study by Rhodes ci a! may cliflbr from 36 the results of a study using diagnostic codes to identify primary MR care in a provincial administrative database.  12  There are no known published studies that have directly compared self-reported primary Mu care to provincial administrative records, using diagnostic codes to identify MR care. Such research is essential because diagnostic codes provide the only method for identifying physician reimbursement records for primary MH care in British Columbia; and PRAC proposes to conduct MH surveillance using MI-I diagnostic codes. In addition, it is important to note that Canadian national health surveys ask individuals if they have talked to a GP about emotional or MH issues, and the question is not conditioned on the modality of treatment rendered (such as psychotherapy). Therefore, when examining agreement between self-reported data versus provincial administrative data, it can he useful to employ the more sensitive definition of MFI care based on diagnostic codes, rather than a more narrow definition, such as one based on service codes.  Main observations  The existing literature is insufficient to estimate differences between self-reported use of primary MR care versus provincial administrative data, using the diagnostic code definition of MH care that is used in British Columbia and that has been proposed by PHAC ft)r national surveillance. The sole published individual-level comparison between self-reported MI-I care and provincial administrative data used Ontario’s MH service codes to identify MFI care in the administrative data; and it has been estimated that the diagnostic code definition of MR care could yield estimates that are twice as high as estimates based solely on service codes.  •  It appears that the use of the diagnostic code definition of MI-I care in the administrative data could result in an overall tendency for estimates of MR care in the administrative data to be higher than estimates in the self-reported data. However, it is important to recognize that the study by Rhodes ci a! did not focus specifically on primary MR care, and that it was conducted in another province, during a different time period, using a different survey instrument, and a different provincial administrative database. Therefore, this assumption must he empirically tested.  •  Based on the information summarized in Table 1 .1. there appears to be a tendency towards low reporting of the number of GP visits for general health issues, hut high reporting of the I .)  number of GP visits for MHissues. The number of self-reported MI-I visits in the present study would be expected to be lower than in the 1 994/95 NPNS because the CCI-IS question refers to the number of MI-I visits to each respondent’s main OP. whereas the NPIIS question referred to the total number of MR visits to all health profssiona1s. It is unknown how the  use of the diagnostic code definition of MN care may affect estimates of the number of GP visits per respondent in the provincial administrative database. The number of visits may he higher because the definition of a MH visit is more sensitive in the present study. However, the study by Rhodes c/ al included all visits to physicians in the administrative data whereas the present study refers only to the number of visits to each respondent’s main OP. The present study will address this information gap.  14  _______  Table 1.1: Summary of three comparisons between self-reported health care and provincial administrative health records Study authors and age characteristics of sample  Study #1 Mustard et a 13 (age 12+)  Study #2 Raina et a1 27 (age 65+)  Study #3 Rhodes et a1 33 (age 12+)  Data sources  Proportion of individuals using services  Number of visits  Provincial administrative data  Self-reported data  Administrative data  1994 NPHS question on use of GP services for general health care  Data from Ontario, British Columbia, Saskatchewan and Manitoba  80.4%  80.1%  7.6%* lower in selfreports  Based on 1994 NPHS question on useofGP services for general health care  Data from Ontario  914%  79.3%  Exact agreement = 13.6% Low reporting 594% High reporting = 26.9%  1994 NPHS question on use of mental health services (from any health professional)  Data from Ontario  8.7%  9.2%  28% higher in selfreports  Self-reported data  *For British Columbia, estimates from self-reported data were 14.5% lower than estimates from administrative data  1.3.3 A specific mental disorder and agreement between self-reported ambulatory MI-I care versus provincial administrative data  Health service researchers commonly use Andersen’s Behavioural Model of Health Service (ise to consider factors that could potentially influence the use of health care, like need (such having a particular illness): predisposing factors (such as age or sex); and enabling (such as income)’ 0  .  as  resources  The model was originally developed in the late 1 960s to integrate  ideas about why families use health care , and it has evolved over the decades. 39  15  Figure 1.2: Factors associated with NIH care use  LriffQ Health system  4’ Predisposing characteristics Enabling characteristics Need  /  USE* OF MENTAL HEALTH CARE  DATA from self-reports  I  [; ,  DATA from provincial administrative records  t  External environment *Model of factors associated with MH care use adapted from Andersen’s Behavioural Model of Health Service Use.  Mi-I researchers are frequently interested in the association between a specilic mental disorder (which may be described as a ‘need” variable in Andersen’s model) and the use of health care’ ’ 8 20,40  The studies described in the previous sections of this literature review revealed that there  can be disagreement between sources of health care data, meaning that some individuals may report care that is not found in their administrative records, and vice versa. II’ individuals with a particular mental disorder are systematically more likely to he classified as primary Ml-! care users by one data source and not the other, then conclusions about the use (or non-use) of primary MH care by individuals who have that disorder may reflect, in part, “misclassification” error rather than actual use depending on what data source is used to measure primary MH care. The use of primary MH care by depressed individuals receives considerable research attention  because of’the striking social and economic burden of depression, and because depression is treatable in primary care 5 settings 8,40-42 There is a need to know if individuals who were ’ depressed in the past 12 months are likely to have had MR care according to both data sources, or only one data source hut not the other, relative to individuals who were not depressed. Yet, research pertaining to depression and agreement between the two sources of data on primary MI-I care is sparse. Raina ef al found no statistically significant association between self-assessed emotional health status and agreement, in their study of Ontario seniors. However, they did not examine depression specifically, and they examined primary health care for any reason, not  16  specifically for  lviii reasons. Rhodes et al found that approximately 45% of the individuals in  their sample who reported having MH care in the past 12 months had an administrative record of a MI-I service during the corresponding 12 month period, and vice versa. From the data presented, it appears that the corresponding percentages for depressed individuals were higher: 79.4% of depressed individuals who had a MI-I service in the administrative record also reported a service; and 59.9% who reported a MR service also had an administrative record of a service. 1-lowever, as noted previously, Rhodes et a! relied on service codes to identify MI-I care. There remains a need to assess whether individuals who were depressed in the past 12 months are more likely than those who were not, to have had primary Mi! care according to both data sources, or only one of the data sources, when diagnostic codes are used to define MH care in the provincial administrative data source.  Main observations  •  Researchers and policymakers are interested in the use of primary MH care by individuals with particular mental disorders, and depression is a common focus of research attention because its societal burdens are widely recognized, and it is amenable to interventions by GPs. If the use of primary MH care by depressed individuals is captured in one data source but not the other, conclusions drawn by studies using different data sources may not he comparable.  •  The results of a previous study suggest that depressed individuals may be  more  likely than  non-depressed individuals to have evidence of MH care in both data sources rather than only one data source, within a 12 month reference period. However, that research did not use diagnostic codes to identify primary MI-I care in the administrative data. There is a need to address this gap.  1.3.4 Methodological gaps in agreement studies  Several factors limit the interpretability and comparability of agreement studies, including a lack of consistent and complete information pertaining to the direction and magnitude of disagreement between data sources, inconsistent definitions of exact agreement, uni -directional or ecologic comparisons, and a lack of common agreement measures. A lack of complete, 17  comparable, and detailed findings was encountered when presenting the study summaries k)r the literature review of this study. The main observations from the literature are summarized below.  Main observations:  •  For dichotomous measures of any use of services (yes/no), it is recommended that Kappa he used as an indicator of the level of beyond-chance agreement. However, because Kappa is 29,  affected by prevalence, it is also advisable to present the proportion of observed agreement  It is also important to present a contingency table for the following reasons: in cases when the prevalence of a phenomenon like the use of MH care is relatively low, observed -  -  agreement may appear high because of the large proportion of “true negatives” even when cases of positive agreement may be low. The contingency table enables the reader to assess the proportion of agreen3ent that is due to positive (yes/yes) agreement and negative (no/no) agreement. The contingency table also enables readers to assess the proportion of selfreports that exceed administrative estimates and vice versa (yes/no; no/yes).  Some studies include standard measures like sensitivity and specificity, and positive/negative predictive values that help the reader to assess agreement in relation to various potential research objectives. However, these measures imply that one data source is a criterion measure. So, when comparing the two data sources in this thesis, it will need to he made clear that the measures are presented to summarize the  information,  but are not meant  to  imply that either data source is a gold standard. By convention, data from administrative records will be temporarily assigned the role of criterion for the purposes of the calculations. •  For the number of visits, weighted Kappa or the Intraclass Correlation Coefficient (ICC) arc standard measures of agreement in the literature. In addition, studies should also provide information about the direction and magnitude of disagreement. The number of health care visits is usually skewed, so it is important to select appropriate summary measures and statistical tests. As it is common for health services research pertaining to the number of visits to refer to the mean number , 25 7 ofvisits 4 ’ 4 the interpretability of the findings is enhanced by  reporting means  in addition to medians. When the number of visits is not 18  normally distributed, it is important to provide a rationale for using parametric or nonparametric tests, or to report the effects of data transformations to normalize distributions br statistical tests.  1.4 A BROADER RESEARCH FRAMEWORK  The objective of this research is to compare two common sources of primary MI-I care data, and to describe differences between them, in order to aid survey methodologists, administrative data analysts, researchers and government agencies, to explore, understand, and potentially ameliorate discrepancies between the two data sources. The research method involves analyses of existing data sources, thus reasons for discrepancies and methods to rn/ligate them cannot appropriately he tested in the present study. Nonetheless, it is useful to present this research in the context of a broader research framework for two reasons:  •  First, when interpreting the results of comparisons between the two types of data sources,  it  is useful to have a framework with which to understand how differences may arise. •  Second, the framework illustrates the potential role of the present study in creating opportunities for hypothesis generation to inform future research by survey methodologists; cognitive scientists; and database designers, administrators, and analysts, to improve data quality in self-reports and administrative data.  In broad strokes, as depicted across the top of Figure 1.3, the process of producing se//reported data for a primary MI-I care visit begins when a patient accesses health care from a GP; then recognizes and acknowledges to themselves that the visit pertained to a MH issue; then is willing  I9  t  The GP recognizes & records a mental health issue  t  The GP submits a billing record with a mental health diagnostic code  The database captures sufficient diagnostic information for research purposes  The process of formulating a MH visit in administrative data...  The GP performs a billable medical service  Respondent is willing and able to report all visits  The wording of the survey question is understood as intended and the response is recorded correctly  I  Source: Factors associated with MH care use are adapted from Andersen’s Behavioural Model of Health Service lJse. 44  External environment  t  Need  Enabling resources  Predisposing characteristics  Health system  Respondent recalls the visit  Respondent acknowledges and recalls that a mental health issue was discussed  The process of formulating a MH visit in self-reported data...  Figure 1.3: The present study within a broader research context  MH CARE DATA  and able to recall and report this information in a health survey; then reports it in the way that was intended by the questionnaire designers; and then the response is recorded correctly. These  processes are in the domain of survey designers. cognitive scientists, and a range of interdisciplinary researchers who study the complex interplay of factors that influence the accuracy of self-reports . Statistics Canada. who produces the CCHS data. employs a vast 4 arsenal of techniques to facilitate high data quality, including questionnaire pre-testing, standardized interviewer training, interviewer debriefings, and well-documented data editing procedures’  Similarly, as depicted across the bottom of Figure 1 .3, the process of producing an adminLcirative record of a MH visit can begin with a GP performing a billable service, recognizing or suspecting that the patient has a MH issue, recording diagnostic information in a medical record, and submitting a billing record with a diagnostic code corresponding to a MEl condition. Provincial administrative databases are designed for financial management purposes rather than MI-I research, and the ability of a MI-I services researcher to extract accurate and  useful information about a visit from the database can depend on aspects of the database design, like the number of data fields used to capture diagnostic information; the precision of the diagnostic codes used; and database administration features, such as audits of key variables.  [hose who design and implement data collection through surveys and administrative databases generally have to assess data quality though internal processes, such as survey pre-testing or data audits, and rarely have the opportunity to link and compare two data sources at the individual level. In summary, it is hoped that the findings of this research will provide evidence with which survey methodologists and administrative data analysts can consider reasons •1() possible differences in the way that MH care use is represented by each data source, with the ultimate aim of optimizing the potential strengths of both data sources for research purposes.  Moreover,  it  is  hoped that, by illuminating the possible direction and magnitude of differences between the two data sources. this research will be informative to researchers and policymakers who design and interpret studies with MH care data, with the ultimate goal of enhancing evidence-based MH policy and practices aimed at improvina the health and well-being of Canadians.  21  1.5 AlMS AND OVERVIEW OF THE PRESENT STUDY  The literature review identified thc following needs:  •  A need to examine agreement between self-reported primary MR care use versus administrative data,  using  MH diagnostic codes to identify primary MR care in the  administrative data. •  A need to examine whether individuals who were depressed in the past 12 months were more likely than those who were not depressed to have had primary MI-I care according to both data sources. or only one of the data sources, or neither data source, using MI-I  diagnostic  codes to identify primary MI-I care in the administrative data. •  A need to examine whether individuals report more or fewer primary MH care visits than are found in their administrative records, using MR diagnostic codes to identify primary MIl care in the administrative data.  •  A need to report comprehensive details of the level, direction and magnitude of disagreement in order to provide relevant information fr a variety of methodological and research audiences; and to use measurement techniques that are consistent with other agreement research in order to facilitate comparisons between studies.  In the course of reviewing the literature, no studies were found that examined agreement between self-reported MI-I care and provincial administrative data for primary MH care specifically. ‘This dissertation focuses on addressing that gap for two reasons. First, GPs are the major source of MH care for Canadians, and thus primary care is the focus of considerable research. and policy interventions, aimed at improving access to care; facilitating the recognition, prevention and appropriate treatment of mental disorders; and optimizing the allocation of medical care resources. Second, unlike psychiatric specialty care, which can he identified in administrative data by the physician specialty code associated with the physician reimbursement record; or in-patient care, which includes multiple diagnostic codes per record, the identiFication of primary MH care in provincial administrative data in British Columbia depends on a single diagnostic code per GP reimbursement record. In the complex environment of primary care, in which patients can present a melange of physical, mental, and social issues, it is useful to compare administrative records to what survey respondents say about whether, and how often, they talked to their GP about their emotions. MI--I or substance use. 22  With these information needs in mind, this dissertation addresses three research questions:  •  What is the proportion of primary MN care users in the sample according to each data source (self-reported data, and administrative data), and what is the level of individual-level agreement between the two data sources? (Chapter 2).  •  Were individuals in the sample who were depressed in the past 12 months more likely than individuals who were not depressed to have been classified as primary MH care users according to both data sources. only self-reports, only administrative data. or neither data source? (Chapter 3).  •  Does self-reported data yield higher or lower estimates than administrative data regarding the number of times that each respondent talked to their main GP about their Ml! in the past 12 months? (Chapter 4).  TI-ic rationale for each question will he described in more detail in each chapter.  The present study will fulfil a pressing need to compare the use of primary MI-I care in self-  reported data versus administrative data. at a time when such research has an immediate audience. In terms of policy relevance, the timing is ideal:  •  To meet expected data demands arising from the release of the Kirby Report and the  creation of Canadian Mental Health Commission, which are expected to generate sustained attention on MH issues; and •  To aid in the interpretation of findings from on-going studies arising from the 2002 CCHS, and the design of the next CCHS on Mental Health and Well-being:  •  To coincide with the development of a proposed national MH surveillance system. which  is currently being assessed by the Public Health Agency of Canada. All of the analyses that will be described within this thesis are based on the creation of a researchable database for the purposes of the research presented herein. The first step towards creating the researchable database involved submitting a Data Access Request (OAR) to the  British Columbia Ministries of Health specifying the data files, data fields. and years of data requested from British Columbia Linked Health Database (BCLI-lD), as well as the variables requested from the Canadian Community Health Survey (cycle 1 .2). Data access requests are facilitated through the Centre for Health Services and Policy Research (CFISPR) and a detailed description of the application and approval process is available on the CHSPR website (http ://www.chspr.ubc.ca/data.  During the CCHS interviews (which were conducted by Statistics Canada), respondents were asked if they would give permission to have their responses shared with provincial health ministries and linked to provincial health data, and if they would be willing to provide their Personal Health Number (PHN). The PHN is a unique identifier for individuals who are eligible for provincial health care services. If a respondent agreed to share and link their data and also provided a P1-IN, their survey responses were linked to their BCLHD records by the British Columbia Ministry of Health.  Under an arrangement with the British Columbia Ministry of Health, the Centre for ILealth Services and Policy Research (CHSPR) at the University of British Columbia maintains the BCLHD for research purposes, and CHSPR provided the data extracts for this study. In addition, the Ministry gave permission for CCHS data to be used in support of this project. The linked data were anonymized with a study identifier replacing the PHN so that no individual could be personally identified, as described in the following chapter. After all of the data sets were received by the researcher, additional criteria for inclusion in the final study sample were assessed. More detailed information about the data linkage and the formation of the final study sample are provided in the following chapter. The next step involved preparing the data for analysis. The initial phase of this process involved checking the accuracy and completeness of the administrative data records, and addressing any observed anomalies in cooperation with CHSPR. Then, decision rules were developed for the creation of study variables that would not only be appropriate for each of the studies described in the following chapters, but that would also be comparable to those used in other studies. This involved a review of the literature and of CHSPR reports; and consultations with staff of CHSPR, the Centre for Applied Research in Mental Health and Addiction (CARMFIA) at Simon  24  Fraser University, and mental health services researchers and analysts in other provinces. The database was then employed to address the research questions in this dissertation.  The same data sources and the same sample were used in all of the studies described in this dissertation. As a result, while the next three chapters of this thesis represent stand-alone manuscripts, the data sources, data linkage, and development of the sample are only described in detail in Chapter 2 in order to avoid repetition for the reader. In the subsequent chapters, the reader is referred back to relevant sub-sections of the Methods section in Chapter 2 for more details.  In some cases, repetition between the chapters is difficult to avoid. For example, some of’ the reviewed literature, and some of the limitations and contributions of this work, apply to all three studies and are described in each manuscript. Where possible, as a courtesy to the reader, later chapters highlight the main points rather than repeating details described in earlier chapters.  25  1.6 REFERENCES  I. Public Health Agency of Canada. conornw burden o/illness in Canada, 1998. Ottawa: Puhhc Health Agency of Canada, Health Canada; 2002. 2. Murray CJ, Lopez AD. Alternative projections of mortality and disability by cause 1990-2020: Global Burden of Disease study. Lancet. 1997;349:1498-1504.  3. World Health Organization. The world health report 200]  --  Mental heal/h: New  understanding, new hope. Geneva: WHO. 2001. 4. Michalak EE, (3oldner EM, Jones W. Oetter UM, Lam RW. The management of depression in primary care: Current state and a new team approach. BCMJ. 2002;44:408-41 I.  5. Wang J, Langille DB, Patten SB. Mental health services received by depressed persons who visited general practitioners and family doctors. Psychia/r Serv. 2003 ;54: 878-883. 6. Mustard CA, God V. Barer ML, Muhajarine N, i)erksen S. Validity of se/f reported utilization  u/insured health care services. Institute/or Work & Health Working Paper #108. l’oronto: Institute for Work & Health; 2000.  7. Watson DE, I-Ieppner P, Roos NP, Reid RJ, Katz A. Population-based use of mental health services and patterns of delivery among family physicians, 1992 to 2001. Can,] Psychiatry. 2005;50:398-406. 8. Bland R. Depression and its management in primary care. Can .1 Psychiatry. 2007;52:75-76. 9. Lin E, Goering PN. The utilization o/physzcjan services/Or mental lieu/i/i in Ontario. Toronto: Institute for Clinical Evaluative Sciences in Ontario; 1999.  26  1 0. Vasiliadis KM. Lesage A, Adair C, Boyer R. Service use for mental health reasons: (‘ross provincial differences in rates, determinants, and equity of access. (‘an J Psych/airy. 2005;50:614-61 9. II Gilbert C. (ice of provincial administrative data/or surveillance of menial disorders: .  Feasibility study. Halifax: Public Health Agency of Canada: 2006.  12. Baldwin W. information no one else knows: The value of self-report. The science o/sc!/ report: Implications for research and practice. 2000:3—7.  13. Urn H. Newman SC’. Bland RC. Design and field methods of the Edmonton survey of psychiatric disorders. Ada Psychialr Scand Suppi. 1 988;33 8:1 7-23. 14. Lefebvre J, Lesage A, Cyr M, Toupin .1, Fournier L. Factors related to utilization of services Ibr mental health reasons in Montreal, Canada. Soc Psychiatry Psych/air Epidemiol. 1998:33:291-298.  15. LinE, Goering P. Offord DR. Campbell D, Boyle MH. The use of mental health services in Ontario: Epidemiologic findings. Can ,J Psych/any. 1996:41 :572-577. 16. Kessler RC. The World Health Organization International Consortium in Psychiatric Epidemiology (ICPE): Initial work and future directions. Ada P.sychiair Scund. I 999:99:2-i). 17. Gravel R, Beland Y. The Canadian Community Health Survey: Mental 1-lealib and Well  being. Can .1 Psychiatry. 2005 ;50:5 73-579. 18. Wang J, Patten SB, Williams JV, et al. 1-leip-seeking behaviours of individuals with mood  disorders. Can J Psych/any. 2005;50:652-659.  27  1 9. Cheung AH, Dewa CS. Mental health service use among adolescents and young adults with major depressive disorder and suicidality. Can .1 Ps ychiati. 2007;52:228-232. 20. Sareen J, Cox B.J, Afifi TO, Yu BN, Stein MB. Mental health service use in a nationally representative Canadian survey. Can J Psychiatry. 2005 ;50:753-76 1.  21. Lin E. Goering P. Offord DR. Campbell D, Boyle MH. The use of mental health services in Ontario: Epidemiologic findings. Can .1 Pychia1iy. 1 996;41 :572-577. 22. Kirby M. Out of the shadows at last: Transforming menial health, mental illness and addiction services in Canada: Final reporl of the Standing Senate Committee on Social A/fairs, Science and Technology. Ottawa: Government of Canada; 2006. 23. World Health Organization, ed. International Classification q/ Diseases, Ninth Revision (ICI)-9). Geneva: WHO; 1977.  24. Bilsker D. Goldner EM. Jones W. Health service patterns indicate potential benefit of supported self-management for depression in primary care. Can] Psychi a/ri’. 2007:52:86-95.  25. Bergeron E, Poirier LR, Fournier L, Roberge P, Barrette G. Determinants of service use among young Canadians with mental disorders. Can .1 Psychiatry. 2005;50:629-636. 26. Streiner DL, Cairney J. Lesage A. Guest editorial: Psychiatric epidemiology in Canada and the CCHS study. Canadian Journal of Psychiatry. 2005;50:57 1-572.  27. Raina P. Torrance-Rynard V, Wong M, Woodward C. Agreement between self-reported and routinely collected health-care utilization data among seniors. Health Serv Res. 2002:37:751774.  28  28. Evans C. Crawford B. Patient self-reports in pharmacoeconomic studies: Their use and  impact on study validity. Pharmacoeconomics. 1999;15:241-256. 29. Ungar WJ, Coyte PC. Health services utilization reporting in respiratory patients. J Cliii Epidemiol. 1998;51 :1335-1342. 30. Rifler PL, Stewart AL, Kaymaz H, Sobel DS, Block DA, Long KR. Self-reports of health care utilization compared to provider records. JClin Epidemiol. 2001 :54:136-141. 31. Cleary PD, Jette AM. The validity of self-reported physician utilization measures. Med ( ‘are. I 984;22:796-803. 32. Glandon GL, Counte MA, Tancredi D. An analysis of physician utilization by elderly persons: Systematic differences between self-report and archival information. JUeronlol. I 992;47:S245-52. 33. Rhodes AE, Lin E. Mustard CA. Self-reported use of mental health services versus administrative records: Should we care? lvii JMeihodc Psychiair Res. 2002;l 1:125-133. 34. Craven MA, Cohen M, Campbell D, Williams 3, Kates N. Mental health practices of Ontario family physicians: A study using qualitative methodology. Can J P.sychlaay. 1997:42:943-949. 35. Steele LS. Glazier RH, Un E, Evans M. Using administrative data to measure ambulatory mental health service provision in primary care. Med Care. 2004;42:960-965. 36. Lin E. Defining a mental health and addictions ‘case’ for national surveillance purposes:  Ontario data. Toronto: Annual Symposium ofthe Canadian Academy of Psychiatric Epidemiologists; 2006.  29  37. Leaf PJ, Bruce ML, Tischler GL, Freeman Dl—!, Jr, Weissman MM, Myers JK. Factors afiecting the utilization of specialty and general medical mental health services. Med Care.  198826:9-26. 38. Vasiliadis HM, Lesage A, Adair C, Wang PS, Kessler RC. Do Canada and the United States differ in prevalence of depression and utilization of services? Psychiatr Serv. 2007;58:63-71. 39. Andersen RM. Revisiting the behavioral model and access to medical care: Does ii matter? .J I-lea//h Soc Behav. 1995;36:l-10. 40. Starkes JM, Poulin CC, Kisely SR. Unmet need for the treatment of depression in Atlantic Canada. Can] Psychiatry. 2005;50:580-590. 41. Collins KA, Westra HA, Dozois DJ, Burns DD. Gaps in accessing treatment for anxiety and depression: Challenges for the delivery of care. C/in Psvchol Rev. 2004;24:583-6 1 6. 42. Motahai R. Diagnosing depression and prescribing antidepressants by primary care physicians: The impact of practice style variations, Ment Health Serv Res. 2002;4: I 09-11 8. 43. Goldberg RW, Seybolt DC, Lehman A. Reliable self-report of health service use by individuals with serious mental illness. Psychiatr Serv. 2002;53:879-881. 44. Tisnado DM, Adams JL, Liu H, et al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Met/Care. 2006;44:132-140. 45. Knox SA, Britt N. The contribution of demographic and morbidity factors to self-reported  visit frequency of patients: A cross-sectional study of general practice patients in Australia. BM( Family Practice. August 2004.  30  46. Conrad FG, Brown NR, Cashman ER. Strategies for estimating behavioural frequency in survey interviews. Memory. I 998;6:339-366.  31  2 USE OF GENERAL PRACTITIONER SERViCES FOR MENTAL HEALTH ISSUES: 4 AGREEMENT BETWEEN SELF-REPORTED DATA AND ADMINiSTRATIVE DATA  2.1 INTRODUCTION  Mental disorders impose substantial personal. social and economic burdens on Canadians. hut  access to timely and appropriate health care interventions can reduce this burden. Most mental health (MH) care in Canada is provided by General Practitioners (GPs)’ , and so inft)rmation 4 about primary MH care ’ utilization is fundamentally important to researchers, policyrnakers and 1 clinicians. This information is used to measure the prevalence of primary MH care, to identify determinants of primary MI-I care, and to develop and assess methods for allocating MI-I care  resources most effectively.  In Canada. the two predominant sources of data on ambulatory MH care are self-reported data and provincial administrative health records. Canada recently conducted its first national MI-I survey, and provincial administrative data are expected to form the basis of a new national MI-I surveillance program in 5 Canada 6 Both types of data are commonly used in the 3 literature 7• 1 If self-reported data and provincial administrative data provide different estimates of primary MH care, it is important for policymakers, planners, and researchers to know to what extent they  may differ in order to select, use, and interpret each measure appropriately. There is a scarcity of research that has directly compared self-reports with provincial administrative health data at the individual level, and the aim of this study is to address this void in the literature. Five studies were found that pertained to this issue. One study compared survey estimates of MH service use from the Epidemiological Catchment Area Project (ECA) in the early I 980s to other data sources. primarily the provider-based Center for Mental l-lealth Service National Reporting Program in the United States’ °. The total number of patients who received ambulatory MH care in the general medical sector was 21% lower in the survey data than the parallel estimates from other sources, hut that study did not directly pertain to how irovtncia1 adminisiraijve  data may compare to self-reports. In a study that examined MI-I  A version of this chapter will be submitted for publication in a scientific journal: Palm J, Koehoorn M, I lertzrnan C, Goldner EM & Zumbo BZ.. “Use of general practitioner services for mentai health issues: agreement between self-reported data and administrative data.” b In this study “primary MH care” refers care provided by GPs.  service use in Ontario’s administrative data. Un and Goering observed that their estimates from administrative data appeared to be substantially higher than those that had been reported in population surveys’  .  However, the findings ftom the ECA study. and the observations by Fin  and Goering were based on ecologic observations and not direct comparisons of the same individuals over exactly the same reference periods.  Few studies have conducted individual-level comparisons, Tauhe el al, conducted a case-by-  case examination of Medicaid data and household survey data from the early l980s  using a  hybrid measure as the criterion, which they referred to as the “best estimate” of service The best estimate included all claims in the administrative data, plus household reports, unless there were unmatched records in both data sources that may have represented the same service, in which case only the administrative claim was included to avoid duplication. Neither the household reports nor the administrative data captured all the MH care users who  were  captured  by the best estimate; hut overall, the household reports were closer than the administrative data to the best estimate. The under-representation of MH care in the administrative data was attributed, in  part,  to the challenges of capturing all services per patient in one administrative  database within a system of multiple providers and payment arrangements.  Golding el a! also conducted individual-level  They compared self-reported Ml I  care from a sub-sample of adults in the Los Angeles ECA study, to each individual’s records in a database of publicly funded local MH services, in order to examine the feasibility of validating survey self-reports. They defined “over-reporters” as individuals who reported using services but had no MH service record in the administrative data. Out of 52 individuals who selfreported MI-i care, 8(15.4%) self-reports matched the administrative data, another 8(15.4%) individuals were possible over-reporters, and the rest could not be confirmed due to missing or amhiuous information which was partly attributed to the existence of multiple providers and payment arrangements. In order to assess “under-reporting,” they created a sample of 348 individuals who did not report using MH services, and who either met the criteria Ibr a measured mental disorder according to the survey instrument (National Institutes of Mental Health [)iagnostic Interview Schedule). or who were among a 1 0% random sub-sample of individuals who did not meet the criteria for a disorder. According to the data presented, 91 .4% had no record of service use in the administrative data, one individual was found to have evidence of service use in the administrative data, and 1 3 cases were unclear, so the authors reported that _5 _)  under-reporting could have ranged from 0.3% to 4.0%. It is difficult to draw conclusions about the net direction of difference between the two measures based on these somewhat ambiguous results.  The review of these four studies highlighted numerous gaps that need to be addressed. Although the latter two studies overcame the limitations of aggregate-level comparisons by conducting case-by-case comparisons, they are difficult to apply to the present context because disagreement between data sources was influenced by the challenges of capturing all  information  about MH services from provider records within a multi-payer system. The data for the ECA and Medicaid studies were collected in the early 1980s and considerable changes have occurred in the recognition and treatment of mental disorders in the decades since then, which could conceivably affect either self-reporting of MH care or the recording of MH conditions by physicians. In addition, the designs of administrative databases have become more sophisticated. Also, the studies that were described generally included MI-I services from a range of medical and non-medical providers in one or both measures, whereas the present study is concerned with primary MR care.  Rhodes et a! overcame several of the limitations of the previously cited studies by comparing self-reported data from a community-based sample (the 1994/95 National Population Health Survey (NPHS)), and provincial administrative data from the Ontario Health Insurance Plan (01-lIP), using individual-level linkages between the two data sources . Respondents to national 14 health surveys in Canada are asked if they would agree to give permission to have their survey responses linked to provincial administrative data. Canada is one of the few countries that have administrative databases that record health service use for virtually all residents within provincial jurisdictions, and data linkages provide a unique opportunity to conduct case-by-case comparisons between self-reported health service use and each respondent’s provincial health administrative records. The study by Rhodes ci al was described in Section 1.3.2 of this thesis, and is described here only briefly in order to avoid repetition for the reader. Approximately 9% of the NPHS respondents reported that they had talked to a health professional about their emotional or mental health in the past year, and a similar proportion of individuals had at least one MH service from a physician according to the OH1P data. From the data presented, among those individuals who had MR care in at least one data source, 29.0% of cases were concordani.  34  Although the study by Rhodes ez’ a! compared self-reported MH care to provincial administrative data, the focus of their study differed in a ftmdamental way from the focus of the present study. As described in Section 1.3.2 of this thesis, there are two methods for identifying MR care in Ontario’s administrative data. MN care provided during general office visits can be identified using  diagnostic codes for MI-I conditions. MR-specific services can be identified using service  codes, and records with such service codes usually include a MH diagnosis as the reason for the . Rhodes ci al focussed on the use of service codes to identify MR care in the 11 visit administrative data, and so individuals who only received MH care during general medical visits were classified as non-users of Mu services for the purposes of their study. It has been estimated that diagnostic codes may capture twice as many MH care users as MI-I service codes . 5 alone’  There is now a pressing need to conduct research that compares self-reported data to provincial administrative data, using the diagnostic code definition of primary MFI care. As noted in Chapter 1, MH diagnostic codes are the only way to identify primary MR  care  in Some  provinces, and the MH diagnostic code definition has recently been proposed as the basis for identifying MI-I records for the proposed national MI-I surveillance system 6  In order to address gaps in the existing literature, this study capitalizes on an opportunity  10  link  self-reported primary MI--I care data to provincial administrative data. The self-reported data are from a sub-sample of the 2002 Canadian Community Health Survey (CCNS 1.2) on Mental l-lealth and Well-being . Each individual’s administrative records are from the provincial 5 administrative health care database of British Columbia. This linked data will he used to address two principal research needs that remain unaddressed by the existing literature. The first objective of the present study is to describe the proportion of individuals who used  primary MH care during a 12 month period according to self-reports and administrative data, and the study will use MH diagnostic codes to identify primary MR care users in the administrative data.  Based on the following factors, it is hypothesized that estimates from administrative data could he approximately twice as high as the estimates from the self-reported data. First, currently unpublished independent analyses of the proportion of primary MR care users, aged 15 and 35  older, in British Columbia’s administrative data suggest that estimates from administrative data could he almost twice as high as the proportion of primary Ml-1 care users that appear in the CCNS .2 Public Use Microdata File for British Columbia, which was 8% (unweighted)  (personal communication with Wayne Jones, Centre for Applied Research in Mental I Iealih and Addiction, Simon Fraser University, May 7, 2007). Second, while caution is exercised when generalizing the findings of Rhodes e/ ,l to the present study because of dilTerences in time periods, MH care delivery systems, sample characteristics, and the inclusion of specialist physicians in their estimates, it is possible that if they had used the diagnostic code definilion, ihe proportion of MH care users may have been approximately twice as high as the 9.2% thund in their study’ . 5  The second objective of this study is to examine agreement between the two data sources, in  order to assess how many individuals were classified as primary MH care users in both data sources; only in provincial administrative data and not self-reports: only in self-reports and not provincial administrative data; or neither data source.  Agreement will he assessed without assuming that either data source is a gold standard, as both have potential limitations. For example, the accuracy of self-reported data depends on the ability and willingness of individuals to recall and report that they had primary MH care. In contrast, while administrative data may not be affected by recall bias, they have other limitations. lor example, the administrative data capture fee-for-service payments, but a small proportion of GP services are provided under alternate payment arrangements that do not generate a reimbursement record for each service and so they would not he recorded in the administrative database. Moreover, in British Columbia. physicians can submit only one diagnostic code per service, which is intended to describe the main reason for the visit, even if multiple issues are discussed. The survey question essentially asks individuals to report any time they talled about their emotions, MH, or use of alcohol or drugs with their GP (regardless of whether or not they received treatment); yet. little is known about the extent to which MR diagnostic codes in the administrative data capture every instance in which a patient discussed an emotional, Ml I or substance use issue with a GP. particularly if such a discussion occurred within the context ola visit that was primarily for a physical issue.  36  2.2 METHODS 2.2.1 Data sources 2.2.1.1 Self-reported data  The self-reported data are from the CCHS, cycle 1 .2, conducted between May 2002 and I)ecernber 2002. The Cd-IS 1.2 included people aged 15 or older in all provinces, excluding residents of the three territories, Indian reserves, institutions, certain remote areas. and full-time  members of the Canadian Armed Forces. The survey has beendescribed in detail elsewlwre. The response rate was 77.7% for the province of British Columbia. In this study sample. 86.2% of interviews were face-to-face, 10.3% were by telephone, and the mode was not stated for 3.5%. A computer-assisted application was used to interview respondents in English and French. in addition, interviews were conducted in Punjahi and Chinese using a list of key words to aid translation by interviewers who were proficient in these languages . Interviews in other 5 languages were not permitted. In this sample, 97.7% of interviews were conducted in English. There were no proxy interviews.  2.2.1.2 Administrative data  The administrative data are from the British Columbia Linked Health Database (BCLHD). which is a collection of linked administrative databases that contain health care utilization data and other types of health information for the provincial population. One of the core databases is the Payment Information Masterfile of the Medical Services Plan (MSP) database. The MSP database contains records of payments by the provincial government to physicians for medical services provided under fee-for-service arrangements. An MSP record includes the date an insured medical service was provided, the specialty of the physician who provided the service, a fee item describing the type of service, and one diagnostic code per service, which is supposed to represent the main reason ibr the service. Information about the origins, use,  format  and  interpretation of the data fields and codes used in the Medical Services Plan data was provided for this study by the Centre for Health Services and Policy Research through resource materials  and responses  to specific queries.  37  During the study time period, approximately 90% o physician services in British Columbia were provided under fee-for-service arrangements; and the rest were provided under alternative . In most cases, fee6 payment arrangements that do not generate patient-level service records’ for-service hilling claims are submitted electronically (using measures that ensure data  security),  and software vendors provide various interface designs for this purpose. Entry of diagnostic information may, for example, be facilitated through the use of drop-down lists which include diagnostic codes and their corresponding labels. In some cases, ftr physicians with low annual hilling amounts, claims are submitted manually. In addition to the data fields noted above, a hilling claim includes the patient’s PHN and name, and the practitioner number and speciality code, and other information. information about the submission, processing and payment of feefor-service claims is available on the Ministry of Health website (http://www.health. gov.bc.cmsp/infoprac/).  The BC LI-ID also includes the Hospital Admission and Separations (HAS) database, I lAS records include information on the dates of admission and separation, and multiple codes for diagnoses and procedures. MSP data and HAS data are recorded in fiscal years beginning April 1. Extracts from two fiscal years (2001/02 and 2002/03) were used lbr this study to overlap with the survey data collection period and the 12 month reference period fbr each individual in the sample, relative to their survey interview date.  2.2.1.3 Data linkage and study sample  During the Cd--IS interview, respondents were asked if they would give permission to have their responses shared with provincial health ministries and linked to provincial health data, and if they would provide their Personal Health Number (PHN). The PHN is a unique identifier for individuals who are eligible for provincial health care services. If a respondent agreed to share and link their data and also provided a PHN, their survey responses were linked to their tIC L[ID records by the British Columbia Ministry of’ Health.  Under an arrangement with the British Columbia Ministry of Health, the Centre for lIealth Services and Policy Research (CHSPR) at the University of British Columbia maintains the BCLI-lI) for research purposes, and they provided data extracts for this study in accordance with strict guidelines to protect the security and confidentiality of the data’ . Any potentially 7 38  identifying information such as names, addresses, the last three characters of each postal code, and day of birth were removed before the data were released to the researcher. PHNs were removed and replaced with unique but anonymous study identification numbers. This  study  received approval from the Behaviour Research Ethics Board of the University of British Columbia (Certificate Number: B03-0750, Appendix C) and data access was granted through an arrangement between the British Columbia Ministry of Health and Statistics Canada.  The full British Columbia sample of the CCI-IS 1.2 included data for 3,902 respondents. The British Columbia Ministry of Health conducted the data linkage. According to the Ministry’s methodology, in order for a consenting individual to be included in the linked sub-sample br this study, there had to be an identical match between the PHN the individual provided during their survey interview and a PHN in the BCLHD. After receiving the linked data, the researcher compared the month and year of birth, and sex, in both the survey data and MSP data and included only those cases for whom there was an exact match on all of these criteria. In total, records for 2,660 individuals (68.2%) met all of these criteria. The MSP Registry file was used to identify linked CCHS survey respondents who were continuously registered in the provincial health care plan for the two fiscal years for which the data were extracted. Of the 2,660 cases in the CCHS linked study sample, 2,383 (89.6%) were continuously registered for both years. The sample was further reduced by 5 survey participants who did not provide a response to the survey question that was used to identify individuals who reported contact with a OP within the past year. This yielded a final study sample of 2,378 individuals. Overall, the study sample included 60.9% of the individuals in the original CCIIS data set for British Columbia. Ihe demographic characteristics of the full British Columbia sample for the CCHS, and the final linked study sample are displayed in Table A.1 in Appendix A.  2.2.2 Definitions: Primary MH care  I)ichotomous measures of primary MH care in the previous 12 month period were created bbr the self-reported data and the administrative data. In the CCHS, respondents were asked if they had ever seen, or talked on the telephone, to any of the following professionals about their emotions, mental health or use of alcohol or drugs during their lifetime: a psychiatrist; family doctor or general practitioner; other medical doctor; psychologist; nurse; social worker, counsellor or psychotherapist; religious or spiritual advisor; or other professional. 39  Next. respondents were asked a series of additional questions about their contacts with each type of professional. The question of interest for this study concerned the last time the respondent saw or talked to a family doctor o OP. Respondents were asked to choose from the following categories: during the past month; between 2 and 6 months ago; between 7 and 12  months  ago:  or more than a year ago (Appendix B). In this study, respondents who reported that their last contact was within any of the first three time periods were categorized as primary Mi-I care users within the past 12 months. Respondents who did not report having talked to UP about a MI-I issue ever, or who reported that the contact was more than a year ago, were categorized as nonusers of primary MI-I care for the purposes of this study.  Using the administrative data, a primary MI-I care user was defined as an individual who had at least one MI-I visit to a OP in the past year. A OP visit was defined as one or more MSP claims for face-to-face GP services for the same patient on the same date. UPs were identified from the physician specialty codes in the MSP database.  MI-I care was identified using diagnostic codes. In British Columbia. each MSP claim includes one diagnostic code per service. Diagnostic codes in the MSP data include ICI)-9 codes, some ICI)-9-CM (Clinical Modification) codes, and some additional codes that are specific to the MSP diagnostic coding system in British Columbia. The codes used to iden1if’ mental disorders in this study were the MSP-specific diagnostic code 50B for “anxiety/depression;” and ICD9 codes for mental disorders (290 to 314, and 316). lCD codes for mental retardation and “speci lie delays in development” were excluded from the definition. This definition closely parallels the definition that has been proposed for national surveillance of’treated” MI-I issues (ICI)9 codes 290 to 319) (personal communication with Charles Gilbert, Public Health Agency olCanada, November. 2007), with the exception of the excluded codes (which were not observed in the administrative records of the present sample of respondents who are from a community survey and who were age 15 and older). These diagnostic codes were also used to examine the principil diagnosis field in the I-lAS data. There were II individuals who were found to have had in patient care, and these individuals also had at least one record of a primary MI-I care visit in the MSP data, so only the MSP data were used for the analyses.  40  2.2.3 Analytic techniques  Frequency distributions were produced to estimate the proportions of primary M[l care users in the self-reported data and administrative data. I)ifferences between the two proportions were assessed by examining the overlap in the 95% confidence intervals. The lack of comparable measures of agreement is a limitation of the existing literature on agreement between sources of data on health care. For the purposes of comparability with other agreement studies, agreement was described in several ways in the present study. As has been suggested in the literature, chance corrected agreement was reported using the Kappa statistic, and overall observed agreement was assessed using the proportion of cases exhibiting agreen1ent . The strength of 820 agreement was described by comparing the Kappa statistic to the Landis and Koch scale : 0.0021 0.20=poor; 0.21 -0.40=fair; 0.41 -0.60=moderate; 0.61 -0.80=substantial; and 0.81-1 .00a1 most perfect. The data were presented in a 2x2 table, and the sensitivity, specificity, positive predictive value, and negative predictive value were calculated, using the administrative data as a temporary gold standard measure for the purposes of the calculations.  The analyses pertain to comparisons between two data sources within a linked sample. iheretore, as in other agreement research, population survey weights were not applied in the . Analyses were conducted using SPSS for Windows (Rd. 12.0.1. 2001. Chicago: 22 analyses SPSS Inc.)  2.3 RESULTS  Ihe study sample included 2378 respondents between the ages of 15 and 96. the mean age was 50 years old, and 55.8% were female. Approximately one-fifth of the individuals reported being single; two-thirds reported that their household income was $30,000 or more; and more than three-quarters reported having completed secondary school. The characteristics of the sample are displayed in Table A. I (Appendix A). The proportion of individuals in this sample who had MI-I care from a GP in the previous year was approximately twice as high in the administrative data (19.3%) (95% CI: 17.8 to 20.9) compared to the self-reported data (8.5%) (95% Cl: 7.5 to 9.7) (Table 2.1). The confidence intervals did not overlap.  41  The four agreement categories that are displayed in the contingency table (Table 2.1) are labelled as follows: (a) Positive Agreement refers to cases in which individuals had primary Mu care according to both self-reported data and administrative data (n report only)  =  151); (b) Disagreement (Self  refers to cases in which individuals had primary MI-I care according to the self  reported data hut not the administrative data (n  =  52); (c) Disagreement (Admin record only)  refers to cases in which individuals had primary MH care according the administrative data but hot the self-reported data (n  =  307); and (d) Negative Agreement refers to cases in which  individuals had no primary MH care according to either data source (n  =  1,868). In the  discussion of the results, individuals in the first three groups are sometimes collectively referred to as primary !vfiI care users because they are considered to have used primary MI! care according to at least one of the data sources.  As depicted in Table 2.1, there was agreement between the two measures for 84.9% of the individuals in this sample, thus overall agreement appeared high. However, most individuals did not have a MI-I visit with a GP in the past year, and so most of the observed agreement in this  sample was accounted for by 1,868 respondents who had no record of a visit in either the selfreported data or the administrative data (92.5% of 2,019 cases of agreement). Of the 510 primary MI-I care users (individuals who had primary MH care according to the survey data and/or the administrative data), there were only 1 51 cases of agreement (29.6% of 5 1 0 cases). The chance corrected agreement for all 2,378 cases was fair to moderate according the Landis  and Koch scale (Kappa  =  0.38; 95% Cl 0.33 to 0.44).  42  ________  ________ _________  Table 2.1: Mental health care from a general practitioner in the past 12 months according to self-reported data and administrative data (n = 2,378), unweighted* Kappa = 0.384 (95% CI 0.332 to 0.436)  Administrative data  Primary MH care  Selfreported data  Primary MH care  No primary MH care  Positive Agreement  Disagreement (Self-report only)  151  52  Total  203  Overall observed agreement: (151 + 1,868)/2,378 = 84,90% Sensitivity**: (151 /458) 0.33  =  (8.54%) No primary MH care  Total  Disagreement (Admin record only) 307  458 (19.26%)  Specificity: (1,868 /1,920) 0.97  Negative Agreement 1,868  1,920 (80.74%)  =  2,175 (91.46%)  Positive predictive value (PPV): (151 /203) = 0.74  2,378 (100%)  Negative predictive value (NPV):(1,868/2,175) = 0.86  Data sources: Linked data from a sub-sample of the Canadian Community Heahn Survey on Mental l-lealth and Well-being (cycle 12) and the British Columbia Medical Services Plan * All data are unweighted because they pertain to individual-level agreement within the sample. **Adrninistrative data were temporarily assigned the role of “gold standard” for the purposes of these calculations.  2.4 DISCUSSION The results of this comparison demonstrate that health care planners and researchers must be aware that self-reported data and administrative data could yield substantially different information about the proportion of individuals who had a MH visit with a GP in the past 1 2 months. The proportion of individuals in this sample who reported talking to their GP about a Mu issue in the past year was very similar to the proportion of individuals in the full CCIIS sample for British Columbia (8.5% versus 8.0%, unweighted). Similarly, the proportion of individuals with a record of a MR visit to a GP in the administrative data was very close to independent (currently unpublished) estimates from the 200 1/02 administrative data in British Columbia for individuals age 1 5 and over (personal communication with Wayne .Jones, Centre for Applied Research in Mental Health and Addiction, Simon Fraser University, May 7, 2007). As expected, the proportion of primary MI-I care users in this study was approximately twice as high as the proportion of NPF-IS respondents who had a MH service forn a physician in the 43  OHIP data according to Rhodes ei a! (9.2%) which was based on a more restrictive definition of a MH visit’ . 4  In terms of positive agreement between the two data sources at an individual level, 74.4% of individuals in this sample who reported primary MH care also had administrative records of’ care (151 / 203). but  only  33.0% of individuals with primary MI-I care according to the administrative  data had self-reported care (1 51 / 458) (See Table 2.1). In contrast, Rhodes ci a!, found that each data source captured approximately 45% of the MR care users who were captured by the other data source . While the two studies cannot he compared directly due to differences in study t4 methodologies, time periods, questionnaires, and provincial MI-I care systems, the differences between the results are congruent with expectations of differences that could occur due to different study definitions of MI-I care’.  By comparing individually-linked records over identical 12 month reference periods, the results of this study strengthened growing evidence that provincial administrative records capture more primary MI-I care users than self-reported data (when diagnostic codes are used to identify primary  MH care in the administrative data). This difference may he attributable to under  reporting by respondents. differences in consumer and provider perceptions of the content of medical visits. or an over-estimation of primary MH care from the administrative data in this  study. Each of these possible factors are described below. It is possible that under-reporting may account for Disagreement (Adinin record only)  ii,  for  example, individuals were to forget that they had discussed their MH with a GP, or ii they were to refrain from reporting MI-I care because of concerns about the stigma associated with menial illness. It is also possible that differences between patient and physician perceptions about the content of a visit may contribute to Disagreement (Admin record only). For example. a palient may present with a constellation of physical complaints of unknown origin, and their GP may record a MH diagnostic code without explicitly discussing a MH diagnosis with the patient. Another possibility is that the number of primary MH care users in the administrative data could  he over-estimated if the range of diagnostic codes used to identify MI-I care users were excessively broad, thereby casting an overly wide net with which to identify MII visits. However, this does not appear to have been the ease in the present study. This study used the 44  same definition that was used in the recent feasibility study for national MFI surveillance, which was agreed upon by a group of MH experts from several provinces in Canada. Moreover. olthe 458 respondents who had at least one MI-I visit, 79.3% were identified  using one  of only tour  diagnostic codes: neurotic disorders (1CD9 300). acute reaction to stress (ICD9 308), “depressive disorders, not elsewhere classified” (ICD9 311), and the MSP-specific code br “anxiety/depression” (SOB). Among these codes, only the diagnostic code ‘50B” is unique to the province of British Columbia. It has been included in the definition of MH care in other research within British Columbia , and the national surveillance feasibility study (personal 4 communication with Wayne Jones. Centre for Applied Research in Mental l-lealih and Addiction, Simon Fraser University, May 7.2007). If 50B were eliminated, this would have slightly reduced the proportion of primary MI-I care users in the administrative data from 19.3% to 17.0°/o. However, there are anecdotal reports that ICD9 code “311” and MSP-•specific code “50B” are sometimes used interchangeably, so it would not have been appropriate lo eliminate this code for the purposes of this study (personal communication with Wayne Jones, Centre br Applied Research in Mental Health and Addiction, Simon Fraser University, May 7, 2007).  Another possibility is that the administrative data could over-represent primary Mu care if UPs or their office staff erroneously recorded MH diagnostic codes on billing records  for  visits that  did not involve MH issues. Although there is a system of checks and balances to minimize overbilling by physicians, the diagnostic codes within billing records are rarely audited. When respondents have more than one MI-I visit with a GP, then a researcher has more confidence that the MH diagnostic codes for those respondents signify the presence of a MI-I issue and are not the result of inaccurate coding. This highlights an intriguing issue. In this sample. individuals with only one MH visit to a UP represented 57.7% of the 307 individuals in the Disagreement  (Admin record only) group. Overall, of the 458 primary MH care users in the administrative data, almost half (48%) had oniy one visit with a MI-I diagnostic code. This proportion of singlevisit users is similar to findings from other research’. If all of the single visit MH care users were removed from the administrative data, then only 10.1% of individuals in the administrative data would he counted as primary MH care users, and the proportion of primary MR care users in the administrative data would not statistically differ from the survey estimate. However, there are several reasons why it would not seem to be appropriate to exclude singlevisit cases from the definition of primary MH care in provincial administrative records. 45  Individuals with single visits represented 57% of those who met the criteria lbr substance dependence only, 37°/a of individuals in the sample with anxiety disorders only, 21% of individuals with mood disorders only. and 22% of individuals with comorbid disorders (mood. anxiety and/or substance use), according to the survey instrument. Omitting single-visit cases from the definition of primary Mil care in the administrative data could affect inferences ahoul associations between primary MH care and different types of disorders, given that single visit cases are not equally distributed across disorders. Moreover, individuals with single visits may have had preventative care, or may have experienced transient psychosocial problems that did not require on-going treatment, or they may have been recent incident cases who would have accumulated more visits after the end of the reference period for this study. Any olthesc scenarios contribute to MR care costs. and such users may he of interest for a variety olresearch questions.  Also, an examination of the raw data revealed that some of the individuals who had single visits in the administrative data reported more than one visit in the survey. So, it is possible that they had more than one visit, hut the administrative data did not capture  some of  their primary Ml!  care visits. This raises an interesting issue that was not central to the present study. hut that  warrants further investigation in future studies. Even if there is agreement that an individual had  any visit, there may be disagreement about the number of visits. There is a need for future research that quantifies the level of agreement for the number of GP visits among individuals who had primary MI-I care according to both data sources, using the diagnostic code dclinition oF MI-I care. Higher numbers of vi sits in self-reports versus administrative data were found by Rhodes e/ al, hut it is unknown if this phenomenon would be observed when using diagnostic codes to count MI-I visits , (This issue will he explored in Chapter 4 of this thesis). 4  In summary. it seems unlikely that administrative data would largely over-represent primary MR care. In fact, it is possible that provincial administrative data may no!  capture  all primary MI-I  care. In this study, if the administrative data were to reflect the survey question exactly, every discussion of a respondent’s emotions, MH or substance use would yield a administrative record with a MR diagnostic code, whether or not the MR issue was the principal or only issue discussed during the visit, or whether or not the physician felt the issue warranted a mental diagnosis. However, as mentioned previously, only one diagnostic code is submitted with each billing record, and the purpose of the diagnostic codes is to provide a reason for each serv ice. 46  and not to track medical histories. MH issues are sometimes discussed during visits that are ostensibly for physical health issues . Thus, if both physical and mental issues were discussed 23 in a single visit, and a diagnostic code were recorded for the physical issue (or for “general  symptoms”) rather than a MH issue, then there would be no evidence in the administrative data that a MI-I issue was also discussed during that visit and it would not he counted as primary ME-I care. Although the recorded diagnostic code is generally considered to represent the most responsible reason for the visit, little is known about how choices are made within physicians’ offices between multiple codes. Steele ci alcompared various combinations of 01-lIP service codes and diagnostic codes in administrative data versus a sample of medical charts in five academic family practice clinics in downtown Toronto. Using the medical chart as the gold standard, they reported sensitivity of 80.7%, and a positive predictive value of 84.9% (they calculated the latter value by estimating that the prevalence of MH care in the clinics was 17.3% of all visits) . The sensitivity was higher (96.1%) when a MH issue was the primary reason thr T ’ 2 a visit. However, the generalizability of these findings outside academic clinic settings, and outside Ontario, is uncertain.  in some cases, there may be reasons for physicians to avoid choosing a MI-I diagnostic code when alternatives are available, such as concerns about stigmatizing the patient. Also, ii is possible that physician training and experience could affect their willingness or ability to assign a mental diagnostic code.  As described above, if MH diagnostic codes were not recorded in billing records for some visits in which MH issues were discussed, that may have contributed to Disagreement (Self report only). In addition, Disagreement (Self report only,) might also have occurred if some primary  care services were not captured in the administrative data. In British Columbia, approximately 1 0% of all payments to physicians are provided under alternative payment arrangements, hut this proportion varies across different geographic areas, settings, and specialties. Estimates of the proportion for this sample cannot be determined from the administrative data because such services do not generate fee-for-service billing records. Also, the survey question specifically asks if the respondent saw or “talked on the telephone” with a GP, but the provincial health care plan does not cover telephone conversations between the doctor and patient, so such contact is not captured in the administrative data. However, it would be rare that a primary care patient  47  would receive all of their MI—I care by telephone, particularly since GPs are not reimbursed kr such care.  Finally, it is possible that Disagreement (Self-report only) could occur because of respondent over-reporting; for example, if a survey respondent who had acknowledged having symptoms of a MR problem during the survey interview felt it was socially desirable to report that they talked to a physician about their symptoms: if they saw another type of health professional but inaccurately report that it was a GP or family doctor; or if they reported MR care that actuall occurred more than 12 months ago (outside the study reference period).  While several possible reasons for disagreement in either direction have been discussed, the objective of this study was to examine whether agreement or disagreement occurred between the two data sources. and the data presented did not provide enough information to explain conclusively why disagreement occurred. Nonetheless, there appears to he little evidence that an overestimation of primary MH care in the administrative data accounted for a large proportion ol the two-fold difference in estimates from the two data sources. It seems more likely that respondent under-reporting may have occurred. This possibility could he explored further using the techniques employed by cognitive scientists and survey researchers to help to identify, understand, and mitigate potential reporting biases. For example, Statistics Canada uses a technique called qualitative interviewing which helps to illuminate how respondents interpret and formulate responses to survey questions. Such research may point to possibilities to improve self-reporting through questionnaire design, question wording or interviewer probing practices. It is also important to recognize that administrative data are not a gold standard measure of primary MR care, and more study would be valuable to reveal how decisions to assign MR diagnostic codes are made (for example, when both physical and MR issues have been discussed in the same visit), and if diagnostic coding practices vary between GPs. Overall, to summarize the findings of the  agreement  analysis, while there was a general tendency  towards Disagreement (Ac/rn/n record only), there were also cases of Disagreement (Selfrepert only). Therefore, the general finding of this study that administrative estimates of primary Ml 1 care exceed estimates based on self-reported data may not always he applicable at the person level. Researchers should he aware that, if the propensity for individuals to he classified as primary MR care users in one data source versus another varies systematically according to 48  demographic or health status characteristics, then this could affect studies of the relationship between these factors and primary MH care use. Rhodes el  cii  examined agreement using  Kappas, and reported that Kappas were heterogeneous for age, distress, and depression. Agreement was lower for older and retired individuals, higher for individuals with high levels of distress in the past month (according to the K6 scale in the 1994/95 NPHS), and higher br individuals classified as depressed in the past 12 months (according to the University of Michigan Composite international Diagnostic Interview short forrn) . However, agreement was 4 not necessarily perfect in such cases. For example, although the Kappa value was higher lbr depressed individuals than non-depressed individuals, 46.5% of depressed individuals reported MI-I care while 34.1% had an administrative record of care. The latter finding merits attention in  light of the importance of understanding the use of primary MI-I care by individuals with depression and other mental disorders in MH policy and services 2 research 25 It is unknown ’ whether or not the two data sources would give similar pictures of the use of’ care by depressed individuals, when diagnostic codes are used to identify primary MH care in provincial  administrative records. Thus, the distribution of individuals with depression and other menial disorders across the four agreement categories should be a priority of future research. (‘l’he association between depression and agreement will be examined in Chapter 3 ol this thesis) Several factors should he considered when generalizing the results of this study to other contexts, or comparing the results to other agreement studies. In this study, the definition of Mil care in the administrative data was meant to reflect the definition in the survey data, which captured any visit in which a respondent talked about their emotions with a GP. Moreover, it was meant to he consistent with the proposed definition for national MH surveillance. Therefore, single-visil cases were not excluded. For certain types of studies with more stringent requirements for case identification, there may be a preference to create a stricter definition of MH in administrative data that requires more than a single MH diagnostic code. in an effort to ensure that the Ml I diagnostic code reflected an actual MH issue and was not the result of coding errors or a suspected MH condition that was later ruled out by physical findings. This may include at least two or more MI-i diagnostic codes, or a MH diagnostic code and some other confirmatory information from a hospitalization record or a pharmaceutical claim, for example. in this study, it was observed that the exclusion of single-visit cases could potentially introduce bias, and thus should be contemplated with this in mind. Ii is also possible that some primary MN care may not he captured by administrative records within a 12 month period, but that it may he observed 49  if the reference period were expanded. As the objective of this study was to compare two sources of data on  primary  MH care over a 12 month reference period, the duration of the  reference periods were not altered for either data source.  The study focussed on primary MR care because GPs provide most MH care in Canada, and because primary MH care is currently a focus of considerable research and policy attention. The identification of primary MH care in administrative data can present unique methodological challenges because MH is one of many health issues addressed by GPs, and it can only he identified in MSP records using diagnostic codes. MR care from psychiatrists, or in-patient hospital stays, could he examined in future studies. There are additional codes for identifying speciality psychiatric services (such as specialty codes and service codes) in the administrative (MSP) data, and there are a wider range of data fields and codes that can he used to detect MR issues in hospital data. It is important to note that psychiatrists in British Columbia are paid under alternative payment arrangements to a larger extent than GPs (personal communication with Wayne Jones, Centre for Applied Research in Mental Health and Addiction. Simon Fraser  University, May 7, 2007) and such payments would not generate MSP records. Inpatient care was rare in this small community sample. Of the II patients who had an overnight hospital stay with a principal diagnosis of a MH condition, 8 had Positive Agreement and 3 had l)isagreement (Adniin record only). Care from other MH professionals, like counsellors, social workers and psychologists is not provided under the provincial medical plan in British Columbia. There is another provincial administrative database (the Client/Patient Information Management System (CPIM)) which provides information about, for example, services received in community Ml! centres, but this database did not include sufficiently detailed information fir the purposes of this study, and only a very small proportion of the study sample appeared in the CPIM data. Thus, these findings should not he generalized to other types of MI-i care without further study. In this study. the term primary” MR care was used for the sake of brevity, to refer to GPs as opposed to specialists. However, the survey question asked if a respondent talked to a (IP or ttmily doctor, and it is recognized that some of the respondents could consider a specialist to he their family doctor, and that individuals may receive primary care from other specialties such as internists. The aim of this study was to match the terms “GP or family doctor’ as closely as possible to the physician specialty code in the MSP data, without speculating about whether or not the respondent may consider a specialist to be a family doctor. Nonetheless, the data were 50  examined to determine whether or not there were specialists who had submitted hilling records with service codes that are normally reserved for OP services or vice versa. Such cases were  rare. and in each case the individual had at least one record of a GP visit with a MI! diagnostic code and a GP service code.  [)ifferences in prevalence should he considered when attempting to compare the Kappa value reported in this study with other agreement studies because the Kappa statistic is affected by prevalence. When the prevalence is quite high or low, agreement due to chance is high and so Kappa tends to he lower. When agreement due to chance is low (for example, as prevalence approaches 0.50), Kappa tends to be higher. Modifications of Kappa have been proposed, such as prevalence and bias adjusted Kappa , but they have not gained widespread use across the 26 agreement literature. Meanwhile, as in other studies, percent agreement scores were reported in addition to Kappa . Similarly, several scales for interpreting the Kappa statistics have been 20 proposed. hut the Landis and Koch scale was used because it was the most ubiquitous in the studies that were reviewed in preparation for this research project’ . Percent agreement and 822 other measures can he affected by prevalence also (for example, percent agreement can be affected by a high proportion of true negatives), so the results were displayed in a contingency table to aid interpretation.  The linked study sample included 60.9% of the complete British Columbia data set. Individuals were excluded if they were not continuously registered in the provincial health insurance plan during the study period, did not agree to a linkage, or were not linked by the British Columbia Ministry of Health because they did not provide their PHN, or the PHN they provided was not accurate. Future studies could be enhanced by conducting probabilistic linkages for individuals who did not provide accurate P1-INs. This technique involves matching personal information (date of birth, sex and postal code) in survey data with corresponding information in the administrative data, so that fewer individuals are excluded from the linkage. As shown in Table A.l (Appendix A) the demographic and mental health characteristics in the complete CCI-IS sample for British Columbia and the linked study sample were similar. In the study sample, there was a slightly higher percentage of respondents in the older age groups, and a lower proportion of nevermarried individuals. There was a slightly lower percentage of respondents who met the criteria for a measured mental disorder or a substance use iss0e in the 51  past year. There may be unmeasured personal, cognitive or motivational characteristics among individuals who did not agree to a linkage or were unwilling or unable to provide an accurate P1-IN that could limit the generalizability of these findings.  This was a community-based sample and so the results may not be generalizable to primary Ml I care use by individuals with severe mental illness. In addition, the results may not be generalizable to other jurisdictions with different MH care delivery systems. For example, the existence of multiple providers that are not captured in a single database, or the presence of systemic incentives or disincentives to record MH diagnostic information in hilling records, could affect agreement between administrative data and self-reports in other contexts. It should also be noted that the computer-aided questionnaire for this study was available in Canada’s two official languages, English and French. In this sample, 97.7% of individuals were interviewed in English. There was only one survey respondent who was interviewed in French and it is unknown whether these findings would be generalizable to that linguistic group. The rest of the individuals in the sample were interviewed in Chinese and Punjabi, via translation. The use of a different interview method for the latter two groups, and potential language barriers for individuals from other language groups within this study population, should he considered by researchers who seek to apply these findings to specific cultural groups. The results of this study suggest several other avenues for future research. First, quantitative research would be very useful to confirm whether or not factors such as mental health status or demographic characteristics are associated with being classified as a primary MR care user in self-reported data only, administrative data only, both sources of data, or neither source of data, (This will be explored in Chapter 3). Second, the direction and magnitude of differences in the number of primary MR care visits need to be quantified. (This will be addressed in Chapter 4). Third, collaborative research by survey designers and cognitive scientists would illuminate if and why survey respondents may under-report their use of primary MI-I care, or in some cases, overreport it. Fourth, it would be useful to use the full provincial dataset to examine physician diagnostic coding patterns and to link this data to descriptive information about physicians (age, gender and years of practice) in order to understand more about MH diagnostic coding practices. Fifth, observations of clinical encounters, medical chart reviews, and interviews with patients, GPs and medical office staff, would he useful to understand more about the content of medical visits and coding practices, in relation to the inclusion (or omission) of MH diagnostic codes in 52  hilling records in British Columbia aiid elsewhere. Sixth, agreement for MI-I care from psychiatrists or other medical specialists, or inpatient hospital stays could he analyzed in Future studies, with a larger sample.  The present study set a foundation for these future avenues of research. It was the first known research project to compare individually-linked self-reported primary ME—I care data From a sub sample of national menial health survey respondents versus provincial administrative dula, using Mi--I diagnostic codes to identify primary MH care. The use of administrative data from a universal health care system minimized disagreement resulting from undetected out of plan service use that may occur in settings with multiple plans. The use of data from a population survey data enhanced the generalizability of the findings because it reflected the actual question wording, question order, and data collection methods, and it was condLLcted in a communitybased sample. It was also the first study to compare self-reported primary MH care data and provincial administrative records in the province of British Columbia.  The lindings of this study have several implications for policy and health services research. The first implication is that administrative data and self-reported data should not he used interchangeably to estimate the prevalence of primary MR care use. The two data sources should not be mixed when conducting comparisons across time periods, or between jurisdictions. Self-reports are currently indispensable for obtaining population-level information about care from non-medical providers, like psychologists or counsellors, whose services are not included under the Medical Services Plan of British Columbia and who do not appear in the provincial MSP database. 1-lowever, if the discrepancy between primary MI--I care in survey dala versus administrative data is primarily due to under-reporting, and under-reporting extends to other types of MR services, then it may not be appropriate to assess service delivery models using  adminisiraiive data to measure primary MH care and se/freports to measure other types of care. A second implication for policyrnakers and researchers is that GPs may be encountering far more MI-I issues than population survey data would suggest, in which case. administrative data vould he an important data source for conveying the need For service delivery models which provide GPs with appropriate support for the management of MI-I patients. Such models may include shared care with specialists, and cost-effective community services or self-managed care . A third and related implication is that access to primary MH care may he better 23 programs”  than self-reported data suggest. However, it is unknown from these findings if primary Mil care is being accessed by those who most need it, so more study on the MR characteristics of individuals who have evidence of care, and those who do not, would be useful. This could he facilitated by investments in the development and accessibility of researchable and linked data files for research and policymaking.  2.5 CONCLUSION  Health policy decisions and research priorities must be based on appropriate information. The results of this study suggest that two different data sources used to estimate the proportion of primary MI-I care users may yield substantially different results. The administrative data yielded a higher prevalence of primary MH care use than the self-reported data. However, this general direction of disagreement between the two data sources was not always applicable at the person level, because it was observed that there was also a small proportion of individuals in this sample who reported having primary MH care hut who did not have primary MR care in the administrative data.  [or aggregate-level estimates of primary MI-I care use, it would be appropriate to use administrative data when seeking to minimize the risks of underestimating the number of primary M F-I care users. However, for analyses that require individual-level primary MR care data, the choice is less clear. Researchers need to he aware that if certain personal characteristics or MI-I issues are associated with being classified as a primary MH care user in one data source hut not the other, the choice of data source used to measure MFI care could affect the strength or significance of associations between those factors and the use of primary MI-I care. Future research must explore these issues.  When selecting a data source, researchers should also consider the general strengths and limitations of each data source. Provincial administrative data offer several benefits. For example, because they are collected routinely, they provide longitudinal perspectives, and can be a low cost alternative to survey data collection. However, administrative data alone otler no information about individuals who need care but who do not come into  contact  with the medical  system. so they have limited applicability for analyses of unmet needs. Surveys offer information pertaining  to potential correlates of care such as demographic details and self  54  perceived need for care, and they include standardized diagnostic instruments, and information a about perceptions of care. Hence, the selection of an appropriate instrument for measuring primary MH care will depend on the study population of interest and on the nature of the research question. Linkages between the two data sources would capitalize on the strengths of both to generate evidence to inform health services research and clinical practice. Thus, it is important that Statistics Canada continue to ask respondents for permission to link their survey responses to other data, and that provincial jurisdictions allow for these linkages to occur for the data sets that they govern, in order to maximize their research potential and investigate questions that could not otherwise be analyzed.  55  2.6 REFERENCES  1. Michalak EE, Goidner EM, Jones W, Oetter HM, Lam RW. The management oldepression in primary care: Current state and a new team approach. BCMJ. 200244:408-4 11. 2. Wang J, Langille DB, Patten SB. Mental health services received by depressed persons who visited general practitioners and family doctors. Psychiair Serv. 2003 ;54:878-883. 3. Liii E, Goering P, Offord DR, Campbell D, Boyle MI-I. The use of mental health services in Ontario: Epidemiologic findings. (‘an J Psychiatry. I 996;41 :572-577. 4. Bilsker D, Goldner EM, Jones W. Health service patterns indicate potential benefit of supported self-management for depression in primary care. Can .1 Psychiatry. 2007;52:86-95. 5. Gravel R, Beland Y. The Canadian Community Health Survey: Mental Health and Well being. Can J Psvchiuliy. 2005;50:573-579.  6. (ii Ibert C. Use  of provincial administrative data fr surveillance a/menial disorders:  Feasibility study. Halifax: Public Health Agency of Canada; 2006. 7. Vasiliadis HM, Lesage A, Adair C, Boyer R. Service use for mental health reasons: (‘ross provincial differences in rates, determinants, and equity of access. Can.! Psychiatry. 2005;50:6l4-61 9.  8. Liii E, Chan B, Goering P. Variations in mental health needs and fee-for-service reimbursement for physicians in Ontario. Psychiair Serv. l998;49:1445-1451.  56  9. Watson DE. l-leppner P. Roos NP, Reid RJ. Katz A. Population—based use ol menial health services and patterns of delivery among family physicians. 1992 to 2001. Can JPsychiatiy. 2005 ;5 0:398-406.  10. Manderscheid RW, Rae DS, Narrow WE, Locke BZ, Regier DA.  Congruence  ol service  utilization estimates from the Epidemiologic Catchment Area project and other sources Arch Gen Psychiatry. 1 993 :50: 1 08—I 14. II. Lin E, Goering PN, Institute for Clinical Evaluative Sciences in Ontario. The utilization o/ physicjan services fir menial health in Ontario. Toronto: Institute for Clinical Evaluative Sciences in Ontario: 1999. 12. Taube CA, Schlenger WE, Rupp A, Whitmore RW. Validity of medicaid household respondent reporting of ambulatory visits for mental disorders. .1 Peon Soc Meus. 1 986:14:243256.  13. Golding JM. Gongla P, Browneli A. Feasibility of validating survey self-reports olmental health service use. Am .1 Community Psychol. 1 988:1 6:39-51. 14. Rhodes AE, [An E, Mustard CA. Self-reported  use of  mental health services versus  administrative records: Should we care? ml J Methods Psychiair Res. 2002;l 1:125—i 33. 1 5. Lin E. Defining a mental health and addictions case’ for national surveillance purposes: Ontario data. Toronto: Annual Symposium of the Canadian Academy of Psychiatric Epidemiologists; 2006.  1 6. Canadian  Institute for  Health Information. The status of alternative payment programs/or  physicians in Canada: 2002-2003 and preliminar’ in/ormation/or 2003-2004. Ottawa: Canadian institute for Health Information: 2005. 57  17. Chamberlayne It, Green B, Barer M, Hertzrnan C, Lawrence W, Sheps SB. Creating a population-based linked health database: A new resource for health services research. Canadian Journal ofPublic Health. 1998;89:270-273. 18. Goldberg RW, Seybolt DC, Lehman A. Reliable self-report of health service use by individuals with serious mental illness. Psychiatr Sen. 2002;53:879-881. 19. Tisnado DM, Adams JL, Liu H, a al. What is the concordance between the medical record and patient self-report as data sources for ambulatory care? Med Care. 2006;44:132-140. 20. Ungar WJ, Coyte PC. Health services utilization reporting in respiratory patienta J Clin Epidemiol. I 998;5 1:1335-1342.  21. Landis JR., Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1 977;33: 159-174.  22. Mustard CA, Goel V. Barer ML Muhajarine N, Derksen S. Validity ofse(f-reported utilization of insured health care services. Institute for Work & Health Working Paper #108.  Toronto: Institute for Work & Health; 2000. 23. Craven MA, Cohen M, Campbell D, Williams], Kates N. Mental health practices of Ontario family physicians: A study using qualitative methodology. Can J Psychiatry. I 997;42:943-949. 24. Steele LS, Glazier RH, Lin E, Evans M. Using administrative data to measure ambulatory  mental health service provision in primary care. Med Care. 2004;42:960-965. 25. Bland R. Depression and its management in primary care. Can J Psychiatry. 2007;52:75-76. 26. Byrt T, Bishop 3, Carlin lB. Bias, prevalence and kappa. JClin Epidemiol. 1993;46:423-429.  58  3 AGREEMENT BETWEEN ESTIMATES OF PRIMARY MENTAL HEALTH CARE iN SELF-REPORTED DATA VERSUS ADMINISTRATIVE DATA AMONG DEPRESSED AND NON-DEPRESSED INDIVIDUALSa  l)epression imposes substantial social and economic burdens on individuals, their families, and ; yet it is commonly reported that between 25% to 70% of depressed individuals do not 2 society’ use mental health (MH) services . As most MR care is provided by General Practitioners 37 (GPs) 8 and depression is treatable in general practice settings, researchers and policymakers ’ 3 are interested in understanding why some depressed individuals receive MH care from GPs and others do not ’ 9  10  In Canada. information about the use of MI-I care is generally obtained from  population health surveys, or from provincial administrative health databases. Provincial administrative databases capture physician reimbursement data for medical services provided to virtually all residents, under Canada’s system of universal health care.  Unfortunately, there is evidence to suggest that these two data sources on which most current understanding of the use of primary MR care in Canada is based, do not provide very similar estimates of the proportion of individuals who use primary MI-I t care ’ . The findings of two studies, which will be described further, have examined agreement between the two types of data sources and have found that a substantial proportion of individuals who reported that they had primary MH care had no administrative record of such care, and vice versa 1  .  For researchers  who design or interpret studies on the use of primary MR care by depressed individuals, such results are disturbing. If some depressed individuals have received MI-I care (“treated depressed”) according to one data source, but are classified as untreated by another data source, then it is important for MI-I services and policy researchers to he aware that the proportion of treated and untreated depressed, and putative factors associated with access to care, could vary depending on the data source used to measure primary MH care. ll’depressed individuals are found to drive the high levels of disagreement between the two data sources. survey methodologists. cognitive scientists, administrative database analysts, and health researchers need to focus attention on what factors may contribute to such discrepancies. i’hus, the aim of A version of this chapter will be submitted for publication in a scientific journal: Palm, J, Koehooni, M, Hertzman, C, Goidner, EM & Zumbo, BZ. “Agreement between estimates of primary mental health care in self-reported data versus administrative data among depressed and non-depressed individuals.” b For the purposes of this study, “primary MH care” refers only to care provided by GPs.  59  this study is to examine whether or not individuals who were depressed in the past 12 months arc more likely than those who were not, to have had MI-I care in a general practice setting according to both data sources, only one of the data sources, or neither data source.  As described in Section 1.3.3 of this thesis. there is scant research on this subject. Raina eta! compared self-reported primarY health care in the previous 12 months versus provincial administrative records among seniors in Ontario’ . Self-reported data and GP reimbursemeni 2 records were linked for each individual, and were compared over identical 12 month reibrenee periods. The researchers did not find a statistically significant association between selfassessed emotional health status and individual-level agreement between the two data sources: however, they did not examine depression specifically. Moreover, their study included primary health care for any reason, not specifically for M[f reasons: and the sample did not include individuals under the age of 65.  Rhodes  ci  al compared self-reported MH care to provincial administrative records of physician  services in Ontario in a study of individuals aged 12 years and older’ . In that study. the self1 reported data referred to any type of health professional, and the administrative records included any type of physician. They found that approximately 45% of the individuals in their sample who reported having MH care in the past 12 months had an administrative record of a Mu service during the corresponding 12 month period, and vice versa. From the data presented, it appears that the corresponding proportions for depressed individuals were higher: approximately 60% of the individuals in their sample who reported having MH care in the past 12 months had an administrative record of a MH service during the corresponding 12 month period, while 79% of depressed individuals who had a MR service in their administrative records reported MI-I care. When interpreting the results of the latter study, it is important to consider the method that was used to define MH care because administrative databases are primarily designed lbr financial management purposes and not research purposes. An administrative record for ambulatory care generally includes a description of the service provided by a physician to a patient; and the reason for a medical visit in the form of a diagnostic code. The codes are assigned by physicians or their office staff when submitting billing records. As described in Section 1.3.2 of this thesis, there are two methods for identifying MI-I care in Ontario’s administrative health records. Rhodes ci a! specifically focussed on service codes rather than diagnostic codes to identify MI I 60  care in the administrative data. and so individuals who only received MR care during general medical visits were classified as non-users of MI-I care services for the purposes  of  their  Ii  study.  has been estimated that Ontario’s MR service codes capture approximately half as many primary MH care users as MH diagnostic codes’ . 3  As described in previous chapters of this thesis. the Public Health Agency of Canada (PIIAC) has recently been assessing the feasibility of conducting national MR surveillance with records from  provincial administrative databases, using MH diagnostic codes’ . Thus, this method of 4  identifying MR care is expected to be used even more widely for research and public health purposes. In addition, some provinces, like British Columbia. do not have MH-speciflc service codes and rely on diagnostic codes to identify primary MH care. So, the present study will address a need to examine whether or not depressed individuals were more likely than nondepressed individuals to have agreement versus disagreement between self-reported primary MIt care data and provincial administrative data, using diagnostic codes to identify’ primary MU care in provincial administrative data. As noted in previous chapters of this thesis, when examining agreement between self-reported data from Canada’s national MR survey versus provincial administrative data, it is useful to employ the more sensitive definition of MR care offered by diagnostic codes. rather than a more narrow definition, like service codes. This is because the survey question asks if the individuals had talked to a GP about their emotions, MI—I or substance  use in the past 12 months, and the response is not conditioned on the mode  of treatment or  service received.  Only one known study has examined agreement between self-reported primary MI-I care and each respondent’s provincial administrative records using diagnostic codes to identify primarv Mt-I care, but it did not specifically examine depressed individuals. That study was the topic of Chapter 2 of this dissertation. As described in Chapter 2, the study was conducted in British Columbia using linked data from a community-based sample. It found that approximately onequarter of individuals who reported that they talked to a GP about their MR in the pasl 12 months did not have an administrative record of a MR visit with a GP during that timeframe: and two-thirds of individuals in the study sample who had an administrative record olsuch a visit did not report that they talked to a OP about their MR. The present study will describe the proportions of depressed individuals in that sample who had primary MI-I care in both data sources, one data source but not the other, or neither data source: and it will investigate whether 61  or not individuals who were depressed in the past 12 months were more likely than those who were not, to have had agreement or disagreement between their self-reported data and administrative records.  The definitions of primary MR care in the self-reported data and in the administrative data are  displayed in Figure 3.1. For the analyses in this study, agreement and disagreement were not conceptualized as dichotomous categories of a single variable because MI-i care is relatively rare in the population and so overall agreement would be expected to be high due to the high proportion of non-users of MR care. Thus, four categories were used to describe agreement and disagreement (Figure 3.1):  •  Positive Agreement occurred when individuals had primary MI-I care according to both self-  reported data and administrative data; •  Disagreement (Selfreport only) occurred when individuals had primary MI-I care according  to the self-reported data but not the administrative data; •  Disagreement (Admin record only,) occurred when individuals had primary MR care  according the administrative data hut not the self-reported data; and •  Negative Agreement occurred when individuals had no primary MR care according to either  the self-reported data or the administrative data.  62  Figure 3.1: Labels for each category of agreement Administrative records Definition  of  primary  An individual had at least one any  ICD9  mental  in  the  health  past  12  Primary MH care  Self-reported data Definition  primary  An  that  they  talked  to  MH  individual  a  on  had  the  GP  diagnostic  or  use  in  the  of  No primary MH care  Disagreement  Agreement  (Self-report  to  only)  seen,  or  telephone,  about  alcohol  *Fl1e definition limited  with  reported  No  their  mental  past  record  code  months*  Disagreement emotions,  care:  service  Positive  Primary MH care  of  care:  MH  GP  12  of  health  or  Negative  primary  MH care  drugs  (Admin  record  Agreement  only)  months*  primary MI-I  depression-specific  care  in  both data  sources  encompassed  any  MI-I  care  from  a  GP and  was  not  care.  I3ased on the data presented by Rhodes et cii, which suggested that more depressed individuals source it is , may have primary MR care in both data sources rather than only one data 1 hypothesized that depressed individuals in this sample would be more likely to have primary Ml I care in both data sources (Positive Agreement) rather than oniy one data source (Disagreement), relative to non-depressed individuals. It is further hypothesized that depressed individuals would  he more likely to have primary MR care in at least one data source than in neither data source (Negative Agreement) relative to the non-depressed individuals in this sample. Thus, the following research questions were investigated with the study sample:  •  Are individuals who were depressed in the past 12 months significantly more likely than nondepressed individuals to have had primary MH care according to at least one data source versus neither data source (Negative Agreement)?  •  Are individuals who were depressed in the past 12 months significantly more likely than nondepressed individuals to have had Positive Agreement (primary Mi-I care in both data  sources) versus either type of Disagreement (primary MR care in the self-reported data only or administrative data only)?  •  Are individuals who were depressed in the past 12 months significantly more likely than nondepressed individuals to have had Disagreement (Self-report only) versus Disagreement (Admin record only)?  Figure 3.2 provides a diagrammatic overview of this study. In this study, ‘depressed” individuals include all individuals who met the criteria for a major depressive episode (MDE) in the past 12 months, according to the World Mental Health version of the Composite International 15 that was adapted for the Canadian Community Health Survey on Mental Diagnostic Interview Health and Well-being (CCHS I .2/WMH-CIDI)’ . All other individuals are classified as non6 depressed, for the purposes of this study. The analysis plan will he described further in Section 3.1.7 (Statistical analyses). As noted in the previous chapters, this will he the first known study to compare self-reported primary MH care data with provincial administrative data, using diagnostic codes to identify primary MH care in the physician reimbursement records. in addition, this study will examine four categories of agreement between self-reported primary MI-I care data and provincial  administrative data, rather than only Iwo categories of agreement (agreement versus disagreement). The rationale for using four categories of agreement is that, for example. the characteristics of individuals who had primary MH care according to both sources of data. versus those who did not have any primary MH care according to either source of data (i.e. most of the population) could differ, and so it is beneficial to examine those two groups as distinct entities.  It is hoped that the findings of this study will lead to a better understanding of possible differences between the two data sources, and that the results can he used to identify avenues for further exploration of discrepancies between the two types of data.  64  Figure 3.2: Logistic regression analyses to assess whether depressed individuals were more likely than non-depressed individuals to be classified in the Agreement groups or I)isagreement groups for primary Mil care use the past 12 months (n = 2,366)  Independent variable: Had a major depressive episode (MDE) in the past 12 months according to the CCHS 1.2/WMH-CIDI (n  =  116)  versus Did NOT have a major depressive episode in the past 12 months according to the CCHS 1.2/WMH-CIDI (n  =  2, 250)  Dependent variables: Four agreement categories pertaining to primary MH care I  ANALYSIS 1: Odds of beg rn NEGATIVE AGREEMENT group versus each DISAGREEMENT group  NEGATIVE AGREEMENT*  versus  POSIT!VE AGREEMENT  NEGATIVE AG RE EM E NT*  versus  DISAGREEMENT (ADMIN RECORD ONLY)  NEGATIVE AGREEMENT*  versus  DISAGREEMENT (SELF-REPORT ONLY)  [ANALYSIS 2: Odds of being in POSITIVE AGREEMENT group versus each DISAGREEMENT group  I  POSITIVE AGREEMENT  versus  DISAGREEMENT (ADMIN RECORD ONLY)*  POSITIVE AGREEMENT  versus  DISAGREEMENT (SELF-REPORT ONL )*  ANALYSIS 3: Odds of being in one DISAGREEMENT group versus the other DISAGREEMENT group  DISAGREEMENT (ADMIN DATA ONLY)*  versus  DISAGREEMENT (SELF-REPORT ONLY)  1)ata sources: Linked data from a sub-sample of the Canadian Community Health Survey on Mental Health and Well-being (cycle 1.2) and the British Columbia Medical Services Plan. *Reference category.  65  3.1 METHODS  3.1.1 Data sources  As described in detail in Section 2.2.1.1, the self-reported data are from a sub-sample of the 2002 Canadian Community Health Survey, cycle 1 .2: Mental Health and Well-being (CCHS 1.2) which was conducted between May 2002 and December 2002. The CCHS 1 .2 included people aged 15 or older in all provinces, excluding residents of the three territories. Indian reserves. institutions, certain remote areas, and full-time members of the Canadian Armed Forces. As described in detail in Section 2.2.1 .2, the administrative data are from the British Columbia Linked Health Database (BCLHD). which is a collection of linked administrative databases containing health care utilization data and other types of health information for the provincial population. Extracts from the Medical Services Plan (MSP) aiid Hospital Admission and Separations (HAS) databases of the BCLHD for two fiscal years (200 1/02 and 2002/03) used fbr this study to overlap with the survey data collection period and the 12  month  were  reference  period for each individual in the sample, relative to their survey interview date. 3.1.2 Data linkage and sample size  The full British Columbia sample of the CCHS 1.2 included data for 3,902 respondents. The data linkage was described in detail in Section 2.2.1.3 of this thesis. In brief in order for an individual to be included in the linked sub-sample for this study, they had to have oFfered consent for a linkage and to have provided their Personal Health Number (PFIN) during their CCHS interview; and there had to be an identical match between their Pl-[N. month and year of birth, and sex, in both their survey data and MSP data. (The researcher did not have access to the confidential PHNs. Rather, the PHNs were replaced with unique but anonymous study identifiers). In addition, they had to have been continuously registered in the provincial health care plan for two fiscal years (in order to capture primary MH care throughout the 1 2 month reference period relative to each individual’s survey completion date). They also had to have provided a response to the survey question that was used to identify individuals who reported contact with a GP within the past year. This yielded a study sample oF 2.378. For the study 66  described in the present chapter, the sample was reduced by 12 individuals (0.5%) due to missing data for the derived MDL variable in the CCHS data set. Thus, the final sample size ftw the present study was 2,366, representing 60.6% of the respondents in the original CCI-IS data file fhr British Columbia.  3.1.3 Key independent variable: Major Depressive Episode (MDE)  In the CCI-IS data file, Statistics Canada created a derived variable that identified individuals who met the criteria for an MDE in the past 12 months according to the CC[IS I .2/WMH-CIDI. The Cd-IS I .2/WMH-CID1 is a lay-administered instrument that is partly coded to the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV)’  .  Detailed information about  the specific survey questions and algorithms used by Statistics Canada to derive the MI)F variable is available in the survey questionnaire and the Derived Variable Specification guide for  the CO-IS 1 .2. This documentation on the Canadian Community Health Survey on Mental Health and Well-being is published on the Statistics Canada website (via )pj!w.statcan.ca/eng1ish/sdds/). In brief, according to that documentation, individuals were deemed to have met the criteria for an MDE within the past 12 months if they met ihe criteria for a lifetime history of major depressive disorder, and had a report of an M[)E in the past 12 months, and had experienced clinically significant distress or impairment in social, occupational or other important areas of functiomng in the past 12 months. The latter criterion could be met  via any of a number of questions, one of which asked if the respondent had received profissional treatment for their feelings during the past 12 months. Due to the nature of the research Ibcus of this study. the analyses were conducted with and without the variable pertaining to that specific question and the results were unaffected.  As in previous research, in the present study  instrument, and not disease-specific  diagnostic codes in the administrative data, were used to identify individuals with 9 depression It . was necessary to rely on the survey instrument because it screened all individuals in the sample For depression. Obviously, the administrative data could not have provided MI-I diagnostic  information for individuals who had no administrative record of a visit to a OP ftr a Ml I issue.  67  3.1.4 Primary MH care  Self-reported data: A primary MH care user was defined as an individual who reported in the CU-IS that they had seen, or talked on the telephone, to a family doctor or GP about their  emotions, mental health or use of alcohol or drugs in the past 12 months.  Administrative data: A primary MH care user was defined as an individual who had at least one MH visit to a GP in the past year. A GP visit was defined as one or more MSP claims Rr faceto-face GP services for the same patient on the same date. GPs were identified from the physician specialty codes in the MSP database.  As described in Section 2.2.2 of this thesis, MI-I care was identified using diagnostic codes. Diagnostic codes in the MSP data include ICD-9 22 codes some ICD-9-CM (Clinical . Modification) codes, and some additional codes that are specific to the MSP diagnostic coding system in British Columbia. The codes used to identify mental disorders in this study were the MSP-specific diagnostic code 50B for “anxiety/depression:’ and the lCD codes for mental disorders (290-314 and 316), excluding lCD codes for mental retardation and specitic delays in development.” This definition is very similar to the definition that has been proposed For national surveillance of “treated” MR issues (lCD codes 290 to 319)14. with the exception of the excluded codes (which were not observed in the administrative records of the present sample of respondents who are from a community survey and who were age 1 5 and older). These diagnostic codes were also used to examine the principal diagnosis field in the I-lAS data. In the study described in the present chapter, there were nine individuals who were found to have had in-patient care, and these individuals also had at least one record of a primary MI-I care visit in the MSP data, so only the MSP data were used for the analyses.  It is important to emphasize that the definition  of primaui’  AffI care in this study encompasses  virtually all diagnostic codes from Chapter 10 (Men/al Disorders,) o/ the J( D-9 manual. ‘I-his closely parallels the “omnibus” definition of MH used by P1-IAC’ . It is not limited only to 4 administrative records which had specific lCD codes for depression. The omnibus definition is used in this study because the CCHS does not specifically ask respondents if they received primary MH care /ör depression in the past year. Rather, it asks respondents about any discussion with a GP about their emotions, MH. or use 01 alcohol or c/rugs in tlie past year.  68  Moreover, it is recognized that depressed individuals may receive primary MW care for other MH issues. Since the aim of this study is to examine whether agreement between two sources of data on primary MH care is associated with having an MDE or not having an M[)E, it does not focus on which specific MHdiagiiostic codes were used to identify care in the administrative data.  3.1.5 I)ependent variables (categories of agreement) and statistical analyses  Frequency distributions were used to report the total number of individuals in each agreement  category, and the number of individuals in each category who met the criteria Ibr an MDL in the past 1 2 months.  Logistic regression analyses were used to calculate unadjusted and adjusted odds ratios and 95% confidence intervals for depression status (MDE versus no MDL) in relation to three dependent variables. As depicted in Figure 3.2, the first analysis compared the Negative Agreement group to the Positive Agreement group and each of the two Disagreement groups. The second analysis compared the Positive Agreement group to each of the Disagreement groups; and the third analysis compared the two Disagreement groups directly with each other. Multinomial logistic regression was used for the multiple-category dependent variables, and binary logistic regression was used for the two-category dependent variable. In each of the three comparisons, depression was assessed alone; then with age and sex; and then with age, sex and total number of (iP visits per respondent according to the MSP data. Age and sex are standard variables, and they could he cross-checked by comparing the linked files. The total number of GP visits (for all reasons)  were included from the administrative data as a measure of overall intensity of primary health care use ’ 17 The total number of GP visits (for all reasons) was not available in the selIreported 12 data. as the CCI-IS 1.2 only asked respondents about the number of visits for Mi! reasons to their main GP. The number of independent variables that could be included in the model was limited by the small number of observations in the Disagreement (Sel/reporI only) category olthe  dependent variable. The study examined the association between depression and agreement between two data sources within a linked sample; therefore, as in other agreement research. population survey weights were not applied in the analyses’ . Analyses were conducted using 8 SPSS for Windows (Rd. 12.0.1. 2001. Chicago: SPSS Inc.).  3.2 RESULTS  The sample included 2,366 indivicuaEs between the ages of 1 5 and 96 years. The mean age was 50 years, and 55.8% were female. One hundred and sixteen (4.9%, unweighted) (95% CI, 4.1 to 5.9) of the individuals in the study sample met the criteria for an MDE in the past 12 months,  according to the CCHS 1 .2/WMH-CIDI. This appears to be slightly lower than the unweighted prevalence of depression in the full British Columbia sample in the CCHS Public Use Microdata File (5.8%. unweighted) (95% Cl, 5.1 to 6.6). The distribution of depressed individuals in the study sample across the four agreement categories is depicted in Figure 3.3. Non-depressed individuals dominated the Negative Agreement group (98%); and also the two Disagreement groups (approximately 90% or more of each Disagreement group); but represented less than 60% of the Positive Agreement group in this sample.  70  Figure 3.3: Number of individuals in each agreement category (n  =  2,366)*  POSITIVE AGREEMENT n = 148 (60 depressed**)  DISAGREEMENT (ADMIN RECORD ONLY)  n = 305 (21 depressed)  DISAGREEMENT (SELF-REPORT ONLY) n = 52 (5 depressed)  NEGATIVE AGREEMENT n = 1.861 (30 depressed)  Data sources: Linked data from a sub-sample of the Canadian Community Health Survey on Menial Health and Well-being (cycle 1.2) and the British Columbia Medical Services Plan. *12 cases were excluded from the linked study sample described previously in Chapter 2 due to missing sellreported information for the variable pertaining to a Major Depressive Episode in the past 12 months. **Depressed individuals were defined as those who met the criteria for a major depressive episode in the past 12 months according to the CCHS I .2/WMH-CIDI.  3.2.1 Negative Agreement versus Disagreement  In this analysis, an odds ratio greater than 1.0 implies that Positive Agreement, [)isagreement (Se/freport only), or Disagreement (.4dmin record only), was more common in individuals vJio had an MDE in the past 12 months versus those who did not have an MDE. relative to the Agreement (Negative) group. As displayed in Table 3.1, the odds of being in the Positive Agreement group, or either of the Disagreement groups (Self report only or ac/into record only) versus being in the Negative Agreement group were higher for individuals who had an M1)F 71  versus individuals who did not have an MDE in the past 12 months. The conclusions did not change when age, sex and total number of GP visits were added to the model.  3.2.2 Positive Agreement versus Disagreement  Odds ratios greater than 1.0 are easier to interpret than those below 1.0. so the results of thc second analysis were reported in terms of the odds of having no depression (no MDE) having depression  versus  (MDE). In this analysis, an odds ratio greater than 1 .0 implies that  Disagreement (Selfreport only). or Disagreement (Admin record only), was more common in those who were not depressed than in those who were depressed, relative to those who were in the Positive Agreement category. As depicted in Table 3.2, the odds of Disagreement (Sel/  report only or admin record only) versus Positive Agreement were higher for individuals who had no MDE in the past 12 months versus those who had an MDE. The conclusions did not change when age, sex and total number of GP visits were added to the model.  3.2.3 Disagreement (Administrative record only) versus Disagreement (Selfrreport onlj’)  As displayed in Table 3.3, the analysis did not show a statistically significant difference in the odds of having an MDE versus not having an MDE when the Disagreement (Sell-report oiili group was compared with Disagreement (Admin record only) group. The conclusions did not change when age, sex and total number of GP visits were added to the model.  72  Li.)  1.00 41.61 (25.55 to 67.77) <.001  P  1.00 39.27  1.00 6.49 (2.41 to 17.48) .001  1.00 5.74 (2.12 to 15.58) .001  <.001  P  N  1.00 5.17  1.00 32.08  (1.90 to 14.09)  (19.13 to 53.80)  No Depression 1.00 1.00 1.00 Depression 4.51 (2.55 to 7.99) J<.O0ij480Jj2.69 to 8.57) <.001 : 4.07 (2.25 to 7.36) Data sources: Linked data from a sub-sample of the Canadian Community Health Survey on Mental Health and Well-being (cycle 1 .2) and the British Columbia Medical Services Plan.  Disagreement (Admin record only) vs. Negative Agreement  No Depression Depression  (23.96 to 64.37)  OR (95% CI)  Disagreement (Self-report only) vs. Negative Agreement  No Depression Depression  Positive Agreement vs. Negative Agreement  OR (95% CI)  Adjusted forage, sex and number of GP visits OR (95% CI)  Table 3.1: Adjusted and unadjusted associations between depression and agreement categories, 2,366)  with Negative Agreement as the reference category, unweighted data (n I Adjusted Unadjusted forage and sex  <.001  <.001  <.001  1.00 6.41 (2.41 to 17.05) <  .001  1.00 6.84 (2.56 to 18.32) <  .001  P  1.00 6.21 (2.31 to 16.71)  Adjusted for age, sex and number of GP visits OR (95% CI)  P OR (95% CI)  Adjusted for age and sex P  Adjusted for age, sex and numberofGPvisits OR (95% CI)  No depression 1.00 1.00 1.00 Depression 1.44 (0.52to4.00) 0.49 1.08 (0.38to3.08) 0.88 1.15 (0.40to3.32) Data sources: Linked data from a sub-sample of the Canadian Community Health Survey on Mental Health and Well-being (cycle 1 .2) and the British Columbia Medical Services Plan.  Disagreement (Self-report only) vs. Disagreement (Admin record only)  OR (95% CI)  Unadjusted  Table 3.3: Adjusted and unadjusted associations between depression and agreement categories, with Disagreement Admin record only) as the reference category, unweighted data (n = 2,366)  Depression 1.00 1.00 1.00 No Depression 9.22 (5.31 to 16.01) < .001 I 8.18 (4.68 to 14.30) i 7.88 < .001 (4.50 to 13.81) Data sources: Linked data from a sub-sample of the Canadian Community Health Survey on Mental Health and Well-being (cycle I .2) and the British Columbia Medical Services Plan.  Disagreement (Admin record only) vs. Positive Agreement  Depression No Depression  Disagreement (Self-report only) vs. Positive Agreement  OR (95% CI)  OR (95% CI) P  Adjusted for age and sex  Unadjusted  Table 3.2: Adjusted and unadjusted associations between depression and agreement categories, with Positive Agreement as the reference category, unweighted data (n = 2,366)  <  <  0.79  P  .001  .001  P  3.3 DISCUSSION  The objective of the present study was to further the investigations of Chapter 2, by examining whether individuals in the same study sample, who had a major depressive episode (MDE) in the past 12 months, were more or less likely than individuals who did not, to have had concordance between their self-report primary MR care data and provincial administrative records, versus discordance. The results suggest that agreement did not occur randomly in the sample; rather, it occurred differentially, depending on whether or not an individual had an MDE during the past 12 months.  The implications of the findings herein are informed by: 1) Odds ratios, which permit comparisons between the groups, regardless of the total size of each agreement category (Table 3.1 to Table 3.3); as well as 2) the number of depressed and non-depressed individuals in each agreement category (Figure 3.3).  Depressed individuals in this sample were more likely than non-depressed individuals to have had primary MH care in both data sources rather than only one data source. Although the data sources were not found to be entirely interchangeable for identifying depressed MH care users, almost all (60 / 65  =  92.3%) of the depressed individuals who reported having primary Ml! care  had a corresponding administrative record; and most (60 / 81  74.1 %) of the depressed  individuals who had an administrative record of primary MN care reported such care. This asymmetry is explained by’ the differences in the denominators which reflect the different sizes of the agreement categories, as illustrated in Figure 3.3 (i.e. Depressed individuals represented approximately 10% or less of each of the Disagreement groups, and the size of the Disagreement (Admin record only) group was much larger (n group (n  =  305) than the Disagreement (Se!/repor/ only)  52)).  The only other known study to compare self-reported MI-I care to provincial administrative data was conducted by Rhodes c/ at, and it involved a different methodology for identifying Mu care in the administrative data, different analytic techniques. and it included a wider range of health professionals Yet, differences between the findings of that study and the present study appear 75  to he consistent with the use of a more sensitive definition of primary MI-I care in the present study (which included any record with a MR diagnostic code and was not restricted to records for Mil services such as psychotherapy).  Another finding of this study was that depressed individuals in this sample were more likely than non-depressed individuals to have had primary MH care in at least one data source (either type of Disagreement) versus neither data source (Negative Agreement). Nonetheless, a notable proportion of depressed individuals in this sample (25.9%) did not appear to have any evidence of primary MI-I care. Non-use of MH care by depressed individuals should continue to he a research priority.  Individuals who experience an MDE are a heterogeneous group who vary in terms of such factors as illness phase, severity, treatment, co-morbid mental disorders, symptom mix, and demographic characteristics. The limited sample size of the current study restricted opportunities to assess whether these factors were associated with the distribution of depressed individuals into particular agreement categories. Research on such factors would he valuable to assess how analyses of primary MH care among sub-groups of depressed individuals could be affected by the data source used to measure care.  One example of such a sub-group includes individuals who met the criteria for an MDE and who also met the criteria for substance dependence or other measured mental disorders. In many health surveys, only depression is measured, but the CC1--TS I .2/WMH-ClI)I also measured  manic episodes, social anxiety, agoraphobia. and panic disorder, as well as substance dependence in the past 12 months’ . A total of 37.9% of depressed individuals in the sample also had at least 6 one of these other conditions, and 22.1% of individuals with other Mi-I conditions also had an  MDE. A post-hoc analysis was conducted to assess whether or not the presence of these other MI-I conditions, besides depression, may have affected the conclusions of the present study. To assess this, a binary variable indicating “another MI-I condition” or “no other M[l condition,” was added to the analytic models. The odds ratios for the key variable (depression) were generally somewhat attenuated hut the conclusions remained the same. Due to the limited sample size. all of the individuals with “other MH conditions” were grouped together for the latter analysis. Yet, mental disorders vary in terms of somatic presentations, symptom duration. treatment, and other factors that could conceivably affect reporting of primary care for MH 76  issues in self-reported data or administrative data. Future research, with a larger sample. should examine specific types of mental disorders in relation to agreement between the two sources of primary MH care data.  In this sample, both Disagreement groups were dominated by the non-depressed: approximately 90% or more of the individuals in each Disagreement group did not meet the criteria for an MDL The two Disagreement groups differed considerably in size, as noted previously, and there were 284 non-depressed primary MH care users in the Disagreement (Admin record only) group. and 47 non-depressed primary MH care users in the Disagreement (Se/f repors’ only group. The Disagreement (Admin record only) group warrants particular attention because of the large number of individuals in this sample who had evidence of primary MH care in administrative records and not self-reported data. On further examination, 86.3% of the individuals in the Disagreement (Admin record only) group who did not meet the crileria Ibr an MI)E, also did not meet the criteria for any measured mental disorders or for substance dependence according to the CCHS I .2/WMI-I-CIDI. While not central to the research questions of this study. this issue was illuminated as a result of the data linkage between the two data sources. This observation raises a concern that survey-based research could substantially understate the number of individuals who receive GP services for MH issues but do not meet the criteria for a CIDI disorder, and may lead to incomplete or misleading perspectives on fietors associated with use of care.  It is important to state that the previous observation is not intended to imply that individuals without a CIDI disorder had no need for care. They may have included individuals who had symptoms of depression or other disorders that did not meet the diagnostic threshold of the survey instrument; those who had other kinds of mental disorders that were not measured in the survey’; those who received preventative or maintenance care; or those with a wide range of psycho-social issues or transient adjustments to stressful events. Needs for care are not strictly delineated by the threshold for meeting the criteria for a mental disorder during a particular timeframe. From another perspective. Kurdyak ci a! suggest that the ClDl may yield inflated prevalence estimates in community settings, so perhaps even some of the individuals who were classified as having an MDE may not have as great a need for MI-I care as others . It is 19 important to understand the needs of all individuals who receive care, so that limited health care resources are aimed at those who can most benefit from them. 77  Further research would be beneficial in order to understand why  SO  many individuals appeared to  have had primary MR care in their administrative records but did not report any primary Mil care in the past 12 months. Quantitative research would be valuable in order to understand more about their symptoms, perceptions, and demographic characteristics. The Latter information is collected in the survey, and data linkages would be invaluable for this kind of research. Qualitative research would also he beneficial, particularly pertaining to individuals in the [)isagreernent (Admin record only) group who did not appear to have a mental disorder. For example. cognitive scientists and survey designers employ qualitative methodologies to examine factors that may affect self-reports, such as aspects of the questionnaire design that prompted recall in some individuals but not others (such as questions about health care that were included  in preceding diagnostic modules for those who met the screening criteria for specitic disorders); and so on. Perhaps some individuals who had transient emotional issues that have resolved by the time of the interview may reframe their past mood states in light of their improved mood state and this may lead to under-reporting of MH care . 20  l)isagreement (Admin record only) may not be solely attributable to respondent under-reporting. it is also possible that, in some cases, individuals may have a suspected MH issue that may later he ruled out or attributed to a physical condition, as suggested by Watson el al . or that some 2 billing records are submitted with inaccurate diagnostic codes. l-lowever, it is probably generally more likely that the administrative data would tend to under-estimate primary MN care than over-estimate it. As noted previously, approximately 10% of medical services in British Columbia are provided under alternative payment plan (APP) arrangements . The proportion of 22 APP services varies across geographic regions, settings, and specialties so it cannot be estimated ihr this sample. and it is unknown if it would have affected depressed or non-depressed individuals in this sample differentially (Personal communication with Sandra Peterson, CiLSPR, June 20, 2006). In addition, since GPs can only include one diagnostic code with each hilling record, if they recorded a diagnosis for a physical symptom or general symptoms, primary Mu care would not be detected according to the study definition, Depressed individuaLs may present with both somatic and psychological 23 symptoms as may other individuals with emotional , issues, and it is unknown from the data presented whether or not this phenomenon would have affected depressed and non-depressed individuals differentially in this sample.  7$  When selecting which data source to use for research on the use of primary Mi! care by depressed individuals, or when interpreting results of studies that have used one data source or another. there are several factors to consider. Depressed individuals who reported primary MR care but did not have an administrative record of care (Disagreement  (Se!freporl only))  represented a very small proportion (4.3%) of the 116 depressed individuals in this sample. This may suggest that when the possible consequences of overestimating the number of depressed primary MI-I care users are of concern (for example, when assessing unmet needs among individuals who had an MDE). then self-reported data may provide more conservative estimates. When the consequences of failing to identify all possible primary MI-I care use for depression are undesirable, then administrative data may be preferable.  The large size of the Disagreement (Admin record only) group and its domination by nondepressed individuals in this sample should be considered in light of its possible eIThcts oh analyses of the relationship between depression and primary MI—I care use. Consider a logistic regression analysis in which depression is the independent variable and primary MFI care is a binary dependent variable (‘Use’ versus Non-use’). lfthe administrative data from this sample were used as the source of primary MH care data, the denominator of the odds ratio would he much higher than if self-reported data were used, and the odds ratio would he attenuated overall. If self-reported data were used, the odds ratio would be amplified. (Refer w Rhodes, .laakkimainen ci al, 2006: and Rhodes and Fung, 2004) 24 Thus, researchers should he aware oF the potential For this sort of differential misclassification bias (however, in this case the “misclassification” is relative to another data source, rather than misclassification relative to observed reality).  The general strengths and limitations of each data source should also be considered, and these were described in Section 2.5 of this thesis. For example, MI-I survey data can provide detailed information about symptoms and cornorhidity that is not available in administrative data. Data linkages could create opportunities to capitalize on positive attributes of both data sources to strengthen data quality, to enhance evidence-based practice, and to inform resource allocation decisions.  The findings of this study should he considered within the context of its scope and limitations. The study focussed on primary M}-1 care because GPs provide most MH care in Canada, and  79  primary MI-I care is currently a focus of considerable research and policy attention. For reasons  described in Section 2.4 of this thesis, these findings should not be generalized to other types of MI-I care without further study.  The study sample only included individuals who provided permission for a data linkage, and who also provided an accurate Personal Health Number. It is unknown to what extent the individuals who provided this information differ from those who did not, in terms of their willingness and ability to provide other information, like information about their use of MH care. FortLmately, the age. sex, and MDE status of individuals in the study sample were similar to those of the full CCHS Public Use Microdata File for British Columbia. As stated in Chapter I.  the proportion of individuals in this sample who reported talking to their OP about a MI-I issue in the past year was very similar to the proportion of individuals in the full CCHS sample for British Columbia (8.5% versus 8.0%, unweighted).  The sample for this study represented 60.6% of the full provincial sample. Larger samples could he obtained by repeating these analyses in provinces with larger samples (such as Ontario), or possibly by pooling provincial data 7 under certain conditions, and such research would also provide information about the generalizabilty of these findings to other provinces. The small sample size limited the number of independent variables that could be examined. In addition, it may have limited the ability to detect a statistically significant difference between the very small  [)isagreernent (Sel/L report only,.) group and the Disagreement (Admin record only) group, This agreement study also had several strengths, which have been described in previous chapters of’ this thesis. It compared self-reported primary MI-I care data from Canada’s only national MI-I survey to provincial administrative data, thereby reflecting the actual survey questionnaire and data collection methods. It was based on individual-level comparisons, and it addressed a need  for research based on the use of diagnostic codes to identify primary MI-I care in the administrative data. In addition, it described four categories of agreement, thereby highlighting the proportion of depressed individuals who had no primary MI-I care according to either data  source (Negative Agreement), and also focussing attention on the MI-I characteristics of the very large group of individuals who had primary MI-I care according to the administrative data hut who may be undetected in survey research (Disagreement (Admin record only)). Such information can be used to set priorities for future methodological and MI-I services research. 80  3.4 CONCLUSiON  Individuals in this sample who had a Major Depressive Episode (MDE) in the past 12 monihs were more likely than those who did not to have Positive Agreement (i.e. MI-I care in both data sources) rather than either type of Disagreement. Further, the depressed individuals were more likely than non-depressed individuals to have had primary MH care in at least one data source versus neither data source (Negative Agreement). These findings demonstrated that the classification of individuals in this sample as primary MI-I care users in ho/h data sources versus only one data source, and in at least one data source versus neither data source, was not a random occurrence; rather it was associated with having had an MDE during the previous 12 months. Although the two data sources were not entirely interchangeable for identifying depressed primary MH care users in this sample, it was encouraging that the administrative data captured almost all of the depressed individuals in the sample who reported having primary MI-I care. and the self-reported data captured most of the depressed individuals who had administrative records of care. The findings also indicated that one-quarter of depressed indi\’iduals in the sample had no evidence of primary MH care in either data source. It was also observed that there was a very  large group of primary MH care consumers who appeared only in the administrative data, and about whose needs for care very little is known. It is important to consider that this study examined use of MH care, hut mere use does not imply that appropriate treatments or preventative interventions were provided. Nonetheless, the accurate measurement of primary MH care use is a foundational step towards the ultimate goui of MH service research which is to appropriately match MH care needs with efléctive treatment.  81  3.5 REFERENCES  1. Murray CJ, Lopez AD. Alternative projections of mortality and disability by cause 1 990-2020: Global Burden of Disease study. Lancel. 1997:349:1498-1504. 2. Collins KA, Westra HA, Dozois Di, Burns DD. Gaps in accessing treatment for anxiety and depression: Challenges for the delivery of care. Cl/n Psychol Rev. 2004:24:583-616. 3. Sareen J, Cox Bi, Afifi TO, Yu BN, Stein MB. Mental health service use in a nationally representative Canadian survey. Can J Psychiairy. 2005;50:753-761. 4. Wang J, Patten SB, Williams .JV, et al. Help-seeking behaviours of individuals with mOod disorders.  Can  JP.sychiaiiy. 2005:50:652-659.  5. Rhodes A, Liisa Jaakkimainen R, Bondy 5, Fung K. Depression and mental health visits to  physicians--a prospective records-based study. Soc Sci Mccl. 2006:62:828-834. 6. Starkes JM, Poulin CC, Kisely SR. Unmet need for the treatment of depression in Atlantic Canada. Can J Psychiatry. 2005;50:580-590. 7. Katz Si, Kessler RC. Lin E. Wells KB. Medication management of depression in the Ii nited States and Ontario. Journal o/Generai Internal Medicine. 1 998:13:77—85. 8. Michalak EE, Goldner EM, .Jones W, Oetter HM, Lam RW. The management of depression in primary care: Current state and a new team approach. BCMJ. 2002:44:408-411. 9. Wang .1, Langille DB, Patten SB. Mental health services received by depressed persons who visited general practitioners and family doctors. Psychiair Serv. 2003;54:878-883.  10. Bland R. Depression and its management in primary care. (‘an .J Psychiatry. 2007:52:75-76. II. Rhodes AE, Lin E. Mustard CA. Self-reported use of mental health services versus administrative records: Should we care? ml i Meihociv Psychiatr Rev. 2002;1 1:125-133. 12. Raina P, Torrance-Rynard V. Wong M, Woodward C. Agreement between self-reported and routinely collected health-care utilization data among seniors. Health Serv Res. 2002;37:751774.  13. Liii E. Defining a mental health and addictions ‘case’ for national surveillance purposes: Ontario data. Toronto: Annual Symposium of the Canadian Academy of Psychiatric Epidemiologists; 2006.  14. Gilbert C. Use o/provincial administrative data/br surveillance u/menial disorders: Feasibility study. Halifax: Public Health Agency of Canada: 2006. 15. Kessler RC. Ustun TB. The World Mental Health (WMH) survey initiative version of the World Health Organization (WHO) Composite International Diagnostic lntervie\ (CIDI). in!!  lvlcthods Psychiair Res. 2004; 1 3:93-121. 16. Gravel R, Beland Y. The Canadian Community Health Survey: Mental Health and Well being. Can J Psychiatry. 2005;50:573-579. 17. Glandon GL. Counte MA. Tancredi D. An analysis of physician utilization by elderly persons: Systematic differences het’een self-report and archival information. .1 Ge;’ontol. I 992;47:S245-52.  0  I 8. Mustard CA, God  \/,  Barer ML. Muhajarine N. Derksen S. Validity 0/sell-reported  utilization of insured health care services. institute fOr  Work  & lieu/i/i Working Paper #108.  loronto: Institute for Work & Heal tli 2000.  19. Kurdyak PA, Gnam WH. Small signal, big noise: Performance of the CIDI depression module. Can J Psychiatry. 2005;50:85 1-856. 20. Aneshensel CS, Estrada AL. Hansell IvLI, Clark VA. Social psychological aspects of reporting behavior: Lifetime depressive episode reports. .J Health Soc Behav. 1 98L28:232-246. 21. Watson DE, Heppner P, Roos NP, Reid RJ, Katz A. Population-based use of menial health services and patterns of delivery among family physicians, 1 992 to 2001. Can J Psvchiuir)’. 2005;50:398-406.  22. Canadian Institute for Health Information. The stains ofallernalive payment irogians  for  physicians in Canada: 2002-2003 and preliminary information/or 2003-2004. Ottawa: 2005. 23. Craven MA, Cohen M. Campbell D, Williams .1, Kates N. Mental health practices of Ontario ftrniIy physicians: A study using qualitative methodology. Can 1 Psychiu1,y. 1997:42:943-949.  24. Rhodes AE, Fung K. Self-reported use of mental health services versus administrative records: Care to recall’? JnijMethods Psychiatr Res. 2004;13:165-175.  84  4 NUMBER OF VISITS TO GENERAL PRACTITIONERS FOR MENTAL HEALTH CARE IN A 12 MONTH REFERENCE PERIOD: A COMPARISON BETWEEN SELF REPORTED I)ATA AND ADMINiSTRATIVE DATAa  Information about the number of times Canadians talk to their Family Doctors or General Practitioners (GPs) about MH issues is used for a wide range of purposes. such as economic analyses, assessments of adherence to treatment follow-up guidelines, and analyses of factors associated with high or low use of MH care . The two predominant sources of data on primary 2 ’ 1 MI-I care in Canada are self-reports and provincial administrative recordsh. Under Canada’s  system of universal, publicly-funded health care, provincial administrative data capture physician hilling records for services provided to virtually all provincial residents. However, little is known about whether self-reported data and provincial administrative data provide similar estimates of the number of MI-I visits that individuals make to their GPs. Individual-level comparisons between these two data sources are important because researchers and policymakers are often interested in the intensity of primary MH care utilization by particular sub-groups of the population. If self-reported data and administrative records yield different numbers of visits at the individual—level, then conclusions may vary, depending on what data source is used to measure MI-I care. and this may lead to inappropriate interventions. Thus, the aim of this study is to provide in formation about whether se/freported data provide higher. lower, or equivalent estimates of primary MI-I care visits by individuals versus provincial administrative data, over a 12 month period in a community-based sample. Previous research has suggested that the number of self-reported visits to MI-I professionals may tend to be higher than the number of visits in administrative data . However, those studies may 4 ’ 3 not be directly generalizable to the aim of the present study for several reasons: they involved aggregate-level comparisons rather than individual-level comparisons; MI-I care in the administrative data may have been underestimated due to out-of-plan care; they included multiple types of MH care providers and did not focus on GPs specifically; and/or they used a diflèrent definition of primary MI-I care than the definition that is proposed for this study. a  A version of this chapter will be submitted for publication in a scientific journal: Palm, J. Koehoorn. M, Hertzman, C, Goidner, EM & Zumbo. BZ. “Number of visits to general practitioners for mental health care in a 2 month reference period: a comparison between self-reported data and administrative data.” For the purposes of this study, “primary MH care” refers only to care provided by GPs.  85  The study that was most applicable to the current study was conducted in the Canadian province of Ontario . In that study. Rhodes el alcompared the number of times individuals (age 12 and 3 older) reported that they had talked to a health professional about their MI-I in the past 12 months versus the number of MH visits to physicians in their provincial administrative health records. using data that was linked at the individual-level. They compared self-reported data from the 1994/95 National Population Health Survey (NPHS) versus administrative records in the Ontario Health Insurance Plan (01-lIP) database. They found a higher number of visits in the survey data  versus the administrative records (the median number of visits in the self-reported data was 2 visits, versus 1 visit in the OHIP data). However, the self-reported data in the study by Rhodes ci cii included all visits to any health professionals, and the administrative data included all visits  to physicians, and so those results may reflect differences that might not be observed in a comparison of GP visits. In addition, Rhodes ci al focussed on the use of service codes to identify MH care in administrative data, Agreement research that uses diagnostic codes to identify MI-I visits in provincial administrative data is currently needed, for reasons described in previous chapters of this thesis.  The objective of the present study is to directly compare the number of self-reported primary MR care visits to the number of visits in provincial administrative records, using diagnostic codes to identify primary MR care in the administrative data. The self-reported data are from Canada’s first national MH survey, and the survey question refers to the number of times the survey respondent talked to the GP they saw most often about their emotions, mental health, or use of alcohol or drugs in the past 12 months. In order to correspond to the survey question, the number of visits in the administrative data will also pertain to the GP seen most often. For brevity, the GP seen most o/len for MR issues will be referred to as an individual’s main GP. For ease of reference, the respondents will he categorized into three groups pertaining to the direction of difference between the administrative data and the self-reported data. Neither measure is considered to be a gold standard; the following labels simply refer to the direction of difference:  $6  •  Low Reporters: individuals for whom the number of visits in the self-reported data was less than the number of visits in the administrative data;  •  Exact Matches: individuals for whom the number of visits  in  the selfreported data was the  same as the number of visits in the administrative data;  •  1-ugh Reporters: individuals for whom the number of visits in the self-reported data was greater  than the number of visits in the administrative data.  The primary research question for this study is whether self-reported estimates of the number of MH visits to each individual’s main OP are higher or lower than estimates from the administrative data (when the diagnostic code definition of MR visits is used to count the number of visits to the main GP in the administrative data). The proportions of individuals who are 1-ugh Reporters, Low Reporters and Exact Matches will also be described. This study  is not  intended to validate either self-reported data or administrative data because neither data source is considered to he a gold standard.  A secondary analysis is proposed for the following reasons. The self-reported data were based on a question that asked about the number of times a respondent talked to their main UP about their problems with their emotions, MR or use of alcohol or drugs; but the wording of’ the question does not imply that an individual’s MR encounter.  was  the principal (or only) reason 1it’ each  In fact, as noted in previous chapters of this thesis. discussions about Ml-! issues are  known to arise during primary care visits that are primarily for physical health problems. Thus, a  patient may have had a visit with a UP about physical symptoms during which they also talked about their emotions, and they may count that visit among their self-reported Mi-I  visits.  However, in some provinces, physicians can record only one diagnostic code per service in a billing record, which is intended to describe the principal reason for the visit. Thus, if a visit involves physical symptoms and MH issues, and the OP submits a diagnostic code for a physical condition or for “general symptoms,” then that encounter would not be counted as a MR visit in the administrative data. As a result it may appear that respondents are “over-reporting” the number of MH visits, when perhaps the administrative data failed to capture all visits in which MR issues were discussed.  In order to address this issue, two analyses were conducted. In the first analysis, the selfreported number of visits was compared to the number of visits to the main UP in the 87  administrative data that were accompanied by a MH diagnostic code. In the second analysis, the self-reported data were compared to the number of visits to the main GP for an type 01 diagnosis. If the number of self-reported MH visits to the main GP exceeds the number of visits to the main GP for any diagnosis, then the difference between the two data sources would not solely he attributable to diagnostic coding issues.  Based on the findings of previous research, it is hypothesized that the number of MI] visits in the self-reported data will he higher than the number of visits in the administrative data, and the proportion of 1-ugh Reporters will exceed the proportion of Low Reporters. It is further hypothesized that the proportion of High Reporters will be reduced by the expansion of the definition of a MI-I visit. However, it is unknown whether or not High Reporting will persist in the latter case.  Like the study by Rhodes ci a!. the present study will capitalize on a rare opportunity to compare self-reported MH care data in a community-based sample, versus provincial administrative records in a single-payer system, using a case-by-case data linkage. The present study will make a unique contribution to the literature by examining primary MH care specifically, and using diagnostic codes to identify MI-I care in the administrative data. It will also he the first study of this kind to conduct a comparison using self-reported data from Canada’s first national MFI survey, and administrative data from the large province of British Columbia. In addition, ii will  he the first known study to examine how agreement is altered by expanding the definition of MI-I visits in the administrative data (by including all visits to the main GP, regardless of diagnosis). 4.1 METHODS  4.1.1 Data sources  As described in detail in Section 2.2.1.1, the self-reported data are from a sub-sample of the 2002 Canadian Community Health Survey, cycle 1 .2: Mental Health and Well-being (CCHS 1 .2) which was conducted between May 2002 and December 2002. The CCI-IS 1 .2 included PeOPle aged 15 or older in all provinces, excluding residents of the three territories, Indian reserves, institutions, certain remote areas. and full-time members of the Canadian Armed Forces.  As described in detail in Section 2.2.1.2, the administrative data are from the British Columbia Linked Health Database (BCLHD), which is a collection of linked administrative databases containing health care utilization data and other types oIhealth information for the provincial population. One of the core databases is the Payment Information Mastertile of the Medical Services Plan (MSP). The MSP database contains records of payments to physicians for medical services provided under fee-for-service arrangements. An MSP record includes the dale an insured medical service was provided, the specialty of the physician who provided the service, a physician billing number, a service code describing the type of service, and one diagnostic code per service, which is supposed to represent the main reason for the service. The BCLHD also includes the Hospital Admission and Separations (HAS) database. The F-lAS database includes information on the date of admission and separation, and codes for diagnoses and procedures. MSP data and HAS data are recorded in fiscal years beginning April 1. Extracts from two fiscal years (200 1/02 and 2002/03) were used for this study to overlap with the survey data collection period and the 12 month reference period for each individual in the sample. relative to their survey interview date.  For  this study. MSP data were used to identify the number of visits to GPs, and each individual’s  main GP; and the HAS data were used to identify individuals with overnight hospital stays. It is unknown if or how individuals may report inpatient stays, when asked specifically to count the number of MB visits to the GP or family doctor they saw most oflen. Previous research excluded individuals with in-patient stays . The HAS data were used to examine the results with 3 and without these cases.  4.1.2 Data linkage and sample size  The fufl British Columbia sample of the CCHS 1.2 included data for 3,902 respondents. The data linkage was described in detail in Section 2.2.1.3 of this thesis. In order for an individual to be included in the linked sub-sample for this study, they had to consent to a linkage and provide their Personal Health Number (PHN): and there had to be an identical match between their P1-IN. month and year of birth, and sex, in both their survey data and MSP data. In addition, they had to have been continuously registered in the provincial health care plan for two fiscal years (in order to capture primary MR visits throughout the 12 month reference period relative to each individual’s survey completion date); and they had to have provided a response to the survey 89  question that was used to identify individuals who reported contact with a GP within the past year. This yielded a study sample of 2,378. For the study described in this chapter, the sample was further was reduced by 6 cases due to missing survey data pertaining to the survey question of interest. Thus, the final study sample (n  :z  2,372) included 60.8% of the individuals in the  original CCHS data set for British Columbia. In order to protect the confidentiality of ihe respondents, the PHNs were replaced with anonymous study identifiers before the data were provided to the researcher, as were physician billing numbers.  4.1.3 Definition of a primary MI-i care visit  —  self-reported data  In the survey, respondents were first asked whether they had ever (in their lifetime) talked to a GP for MI-I reasons:  “Have you ever seen, or talked on the telephone, to any of the following professionals about your emotions, mental health or use of alcohol or drugs: Psychiatrist; family doctor or general practitioner; other medical doctor such as a cardiologist, gynaecologist or urologist; psychologist; nurse; social worker, counsellor or psychotherapist; religious or spiritual advisor such as a priest, chaplain or rabbi; other professional.” Respondents who reported having ever seen or talked on the telephone with any of the professionals on the list were then asked a series of additional questions about their contacts with each type of professional. The series of questions about health care from a family doctor or GP began with this statement:  “You mentioned that you saw, or talked on the telephone, to a family doctor or general practitioner about your emotions, mental health or use of alcohol or drugs.”  The question of interest for this study concerned the number of times the respondent talked to a GP in the past 12 months (Appendix B). As noted previously, the CCHS did not ask about the total number of visits to all GPs for MI-I reasons, rather it referred to the UP seen most of/en by the respondent in the past 12 months:  90  Think of the family doctor or the general practitioner you talked to the most often during the past 12 months.  How many times did you see, or talk on the telephone, to this family doctor or general practitioner (about your problems with your emotions, mental health or use of alcohol or drugs)?”  4.1.4 Definition of a primary MH care visit  —  administrative data  A primary MI—I care user was defined as an individual who had at least one MFI visit to a GP in the past year. A visit was defined as one or more MSP claims for GP services for the same patient on the same date. GPs were identified from the physician specially codes in the MSP database. Mi-I visits were identified using diagnostic codes for mental disorders. Diagnostic codes in the MSP data include ICD-9 codes, some ICD-9-CM (Clinical Modification) codes, and some additional codes that are specific to the MSP diagnostic coding system in l3ritish Columbia. As in the other chapters of this thesis. the codes used to identify mental disorders were the MSP-specific diagnostic code 50B for ‘anxiety/depression;” and lCl) codes 29() to 314, and 316. Anonymized physician identification numbers were used to determine which OP was seen most often by each respondent. and to count the number olvisits to that OP.  4.1.5 Statistical analyses  The median, mode and maximum were calculated to describe the number of MI-I visits to the individuals’ main GPs according to the MSP data, and according to the self-reported data. ‘Ihe nun-iher of OP visits is generally skewed to the right in the general population.  SO  medians  are reported in this study. Nonetheless, it is common for studies that refer to the number of physician visits to report means . So. for the purposes of comparability with other studies, 7 means are also reported in this study.  The principal analyses compared the number of MH visits to the main OP in the sell-reported data versus the administrative data for individuals who had at least one visit in ho/h data sources (Group A in Figure 4.1). By definition, individuals in Group B administrative data but zero visits in the self-reported data  —  —  who had at least one visit in the  were Low Reporters. Similarly, 91  individuals in Group C administrative data  —-  who had at least one visit in the self-reported data hut zero visits in the  were High Reporters. Descriptive information was provided about the  number of MN visits to the main GP in the administrative data (Group A  -F  Group B) and the  self-reported data (Group A + Group C) I’or reference. hut the focus of the study was on comparing non-zero numbers of visits in one data source versus the other (Group A, n  145).  Figure 4.1: Number of individuals in the study sample, and in the study sub-sample for the principal analyses LINKED STUDY SAMPLE  n  =  2,372*  SUB-SAMPLE FOR PRINCIPAL ANALYSES GROUP A At least one MH visit to a GP in both the administrative data and the self-reported data  GROUP B n = 307 / At least one MH visit to a GP in the administrative data but no visit in the selfreported data  Individuals with no MH visits to a GP n = 1,868  GROUP C n=52 At least one MH visit to a \ GP in the self-reported data but no visit in the administrative data  Data sources: Linked data from a sub-sample of the Canadian Community Health Survey on Mental Health and Well-being (cycle 1.2) and the British Columbia Medical Services Plan. *6 cases were excluded from the linked study sample described previously in Chapter 2 due to missing selF-reported information about the number of GP visits for MH issues.  The following methods were used to conduct comparisons for Group A: It was appropriate to use the Wilcoxon signed rank test. a non-parametric alternative to the paired t-test, to compare the median number of visits in each data source I)ue to the .  propensity to report means in studies that examine the number of physician visits, some agreement studies have compared means using paired t-tests. So. in this study, paired t-Eesl were also conducted on log-transformed data and non-transformed data. All tests yielded consistently statistically significant results (P <0.001). Thus, only the results from the Wilcoxon signed ranL lest are reported.  92  The difference between the number of MH visits to the main GP in the MSP data was subtracted from the number of visits in the self-reported data for each individual who had at least one visit in both data sources, and the individuals were grouped into three categories: Fligh Reporters, Low Reporters and Exact Matches. 1ie Chi-squared test of equal proportions was used to  compare the proportion of High Reporters and Low Reporters. Descriptive information was provided about the magnitude of High Reporting and Low Reporting (using mean diflèrences for ease of interpretability). In addition, as in other research that has compared the number of physician visits in two data sources, the Intraclass Correlation Coefficient (ICC) was calculated, and the strength of agreement was described by comparing the ICC to the Landis and Koch scale: 0.00-0.20 0.81-1.00  =  poor; 0.21-0.40  almost perfect’°’  =  fair; 0.41-0.60  moderate; 0.6 1-0.80  =  substantial; and  The proportions of High Reporters, Low Reporters and Exact  Matches were also calculated for a second scenario, in which the difference between the number of visits to the main GP (for any reason) in the MSP data was subtracted from the number of MI-I  visits to the main GP in the self-reported data for each individual.  Descriptive information about the diffi?rences between the number of MH visits to the main GP in the self-reported data versus the MSP data were summarized. Differences in opposite directions can cancel each other out, thus obscuring the true magnitude of differences between two data sources, so absolute differences were reported as well as observed differences. In  addition, summary statistics were provided to describe the number of visits to the main GP (lhr any reason) and the number of MI-I visits to all GPs, in the administrative data. (The CCI-IS did  not ask respondents for that information so it was not available in the self-reported data). The analyses pertain to comparisons between two data sources within a linked sample. Iherefore, as in other agreement research, population survey weights were not applied in the . Analyses were conducted using SPSS for Windows (Rd. 12.0.1. 2001. Chicago: 2 analyses’ SPSS Inc.)  4.2 RESULTS  The ages of the 2,372 individuals in the full study sample ranged from 15 to 96 years. The mean age was 50 years old, and 55.8% were female. According to the World Mental I-iealth version of the Composite International Diagnostic Interview that was adapted for the CCHS (CCHS  1 .2/WM1-1-ClDI)’ 3  11 .8% of these individuals had a major depressive episode. manic episode.  agoraphobia, panic disorder. social phobia. or substance dependence (alcohol or illicit drugs) in the past 12 months. Among the 145 individuals who were examined in the principal analyses. the ages ranged from 1 5 to 90 yeus. The mean age was 44 years, 63.4% were female and 56.6% had a measured mental disorder or substance dependence according to the CCI IS I ,2/WMH-CIDI.  As described in Table 4.1, there was little difference in the total number of MI-I vlsi/s to each individual’s main OP in the administrative data (1.045 visits) versus the self-reported data (965 visits) for the full study sample of 2.372 individuals. However, because the number of individuals who had primary MH care in each of the two data sources was lower in the selfreported data (197) versus the administrative data (452), the median, mean and maximum number of MN visits to the main OP were higher in the self-reported data than the administrative data.  When the number of visits to the main GP in the self-reported data versus the administrative (iata were compared for the 145 individuals who had a non-zero number of visits in ho/h data  sources,  the self-reported data also yielded a higher median number of MH visits to the main OP than the administrative data (3.0  visits  versus  2.0 visits), and this difference was statistically significant.  The means were also higher in the self-reported data versus the administrative records (5.4 visits versus 3.4 visits). In addition, the proportion of these individuals who had four or more visits was higher in the self-reported data than in the administrative data (46.2% versus 20.7%). At an individual level, the median difference was 0, and when the effects of sign were removed the absolute median difference was 1.0. As depicted in Table 4.2, only 26.9% of individuals who had at least one visit in both data sources had exact agreement between the number of visits to their main OP in the sell-reported data versus the number of visits to their main GP in their administrative records (Exact Matches). The Intraclass Correlation Coefficient was 0.450 (95% Ci, 0.285 to 0.583), which represents fair to moderate agreement on the Landis and Koch scale . 10  94  ____  -  Table 4.1: Number of visits to General Practitioners in the past 12 months in the selfreported data and administrative data Number of MH visits to main GP in the past 12 months for individuals who had at least one MH visit in either data source At least one MH visit in At least one MH visit in the administrative data self-reported data (n=452) (n197) (Group A + Group C*) (Group A + Group B) Total number of MH visits to main GP in the sample Median** Mean Mode Maximum  1,045 1.0 2.3 1.0 44.0  965 3.0 4.9 1.0 52.0  Number of MH visits to main GP in the past 12 months for individuals who had at least one MH visit in both data sources (Group A only) Administrative data Self-reported data 2.0 3.0 3.4 5.4 1.0 1.0 44,0 52.0  MH visits to main GP (n = 145) Median** Mean Mode Maximum of individuals % with 4+ MH visits to main GP 20.7% 462% DIFFERENCE = the number of MH visits to the main GP in the self-reported data minus the number of MH visits to the main GP in the administrative data, per individual (n = 145) Median 0.0 Mean 20 j Maximum 42.0 j ABSOLUTE DIFFERENCE = the number of MH visits to the main GP in the self-reported data minus the number of MH visits to the main GP in the administrative data, per individual, regardless of sign (n = 145) MecFan 1.0 Mean 3.5 Maximum 42.0  -  —  —  1  All MH visits to all GPs (n = 145) Administrative data Self-reported data Median 3.0 Mean 4.0 This information was not Mode collected in survey 1.0 Maximum 61.0 All visits (for jy reason) to all GPs (n = 145) Median 8.0 Mean 1 1.0 This information was not Mode 4.0 collected in survey Maximum 103.0 I)ata sources: Linked data from a sub-sample of the Canadian Community I-lealth Survey on Mental Health and Well-being (cycle 1.2) and the British Columbia Medical Services Plan. *Refer to Figure 41. **statistically significant difference between medians (P < 0.001). —  -____  --_____  —  —___________  95  Table 4.2: Direction of difference in the number of mental health visits to respondents’ main General Practitioners in the past 12 months in the self-reported data versus provincial_administrative health records (MSP) (n 145) Scenario I  Scenario 2  Number of visits to main GP in self-report ed data (for MH reasons) versus number of visits to main GP in the administrative data (for MH reasons)  Number of visits to main GP in self-reported data (for MH reasons) versus number of visits to main GP in the administrative data (for ANY reason)  C  (I)  ‘CD  Direction of disagreement between self-reported data and administrative (MSP) records  c  o  -  9—>  0 L ()  czt ii  c E  C  U)  U)  Cl)  .4-  ‘-  ( G)C  C  -  0  z  0  Self-reported estimates 49.0%* 5.7 71 24.9 % exceed estimates Self-reported Exact estimates 0.0 39 26.9 51 0 Matches equal MSP_estimates Self-reported Low estimates are 3.1 35 24 1 24.1 Reporting lower than MSP estimates Data sources: Linked data from a sub-sample of the Canadian Community Health Survey on Mental l-leal(h and Well-being (cycle 1.2) and the British Columbia Medical Services Plan. *Statistically significant difference between the proportion of High Reporters and Low Reporters (P <0.001). **ln Scenario 2, for 55.6% of the High Reporters, the self-reported number of visits to their main GP (ihr MH reasons) exceeded the total number of visits (for any reason) to all GPs in their administrative records. High Reporting  **  There was a significantly higher proportion of High Reporters (49.0%) than Low Reporters (24.1%), as shown in Table 4.2 (Scenario 1). The magnitude of the differences was not trivial: the High Reporters had a mean difference of 5.7 more visits in their self-reported data than their administrative records, and the Low Reporters reported 3.1 fewer visits than were ft)ufld in their administrative records.  In Scenario 1, MI-I visits to the main GP included only administrative records that were accompanied by a PJH diagnostic code. On average, administrative records with Mu diagnostic codes comprised 56% of all administrative records of visits to the main GP of each respondent.  96  In Scenario 2, the number of “possible” MN visits to the main GP in the administrative data was expanded to include all visits to that GP, regardless of the diagnostic codes associated with the visits. In the latter case, the proportion of 1-ligh Reporters decreased from 49.0% (95% Cl, 40.6% to 57.4%) to 24.9% (95% Ci, 18.2% to 32.8%), and there was a corresponding increase in the proportion of Exact Matches. For the remaining High Reporters, the number of self-reported visits to their main GP was compared to the total number 0/visits to all GPs/br any reason in their administrative records. In more than half of those cases, the self-reported estimates exceeded the total number of visits to all GPs for any reason in their administrative records. The proportion of Low Reporters (24.1%) was the same in Scenario I and Scenario 2 because I he number of visits in the administrative data exceeded the self-reported number of visits for those individuals under both scenarios.  4.3 DISCUSSION  The purpose of this study was to compare the number of MH visits made to GPs according to self-reported data versus provincial administrative health records. The source of the selfreported data was the CCHS, and the survey question referred only to the GP seen most often by the respondent. A corresponding measure of MR visits to the GP seen most often was constructed from the provincial administrative health data for the analyses in this study. Approximately half of the individuals in the sample reported more MN visits to their main GP than were found in their administrative records. This finding of a tendency towards i-ugh Reporting of the number of MN visits is consistent with previous research that compared selfreported Mi-I care data with provincial administrative records, although parallels arc drawn cautiously due to differences in study methodologies . 3 The reasons for discordance cannot be determined from the data presented; however, several observations and recommendations for future research are offered. 1-ugh Reporting in this study could be the product of several possible phenomena including an under-representation of MN visits in the administrative data, over-reporting by survey respondents, or di//ring perceptions by physicians and patients. Each of these possible factors are described below.  97  It is possible that an underestimation of MH visits in the administrative data (relative to the selfreported data) could have occurred if: (a) the fee-for-service billing records in the MSP data did not include a IvJH diagnostic code!ör every GP visit in which emotional issues were addressed; or (b) the administrative data did not include hilling records for all of/he OP visits that were reported by respondents. Primary care patients present with a myriad of physical, emotional, social. and personal issues, sometimes within the same visit 1  .  Moreover, physical symptoms  may arise from MH issues and vice versa . However, because physicians can only record one 1 diagnostic code per visit, a GP could conceivably record a code for a MI-I issue, a physical condition, or a non-specific code for general symptoms for a visit in which an emotional issue was discussed. It also seems plausible that since the patients had evidence of at least one N/IF! visit according to both data sources, then a MR issue may have been mentioned during other visits, even if it was not the principal reason for those visits and was not reflected in the billing record.  Although the administrative records could not be compared to medical charts for the presen[ study, Steele c/ al conducted a study that compared provincial administrative records to medical charts for 882 unique patient visits in five academic family practice clinics in downtown . In approximately 19% of cases, a MH issue that appeared in the medical chart did not 16 Toronto appear in the administrative record. This was reduced to 4% ol cases when a MH issue was the primary reason for a visit. It must be noted that the applicability of their findings outside academic clinic settings, and outside the Ontario health care system is uncertain, and that the study relied on the content of medical charts in lieu of direct observations of clinical encounters. The second analysis in this study considered the scenario in which respondents were reporting accurately, but the billing records for some of their “mental health” visits were assigned nonmental diagnostic codes. That analysis expanded the window of “possible” MIT visits to include all visits to the main GP, not only visits with a MH diagnostic code. Even this generous definition of “possible” MH visits did not completely eliminate High Reporting. One-quarter of the individuals continued to be High Reporters. Moreover, among the remaining 1-ugh Reporters. there were some individuals (representing approximately I4°/ of the study sub sample) for whom the number of MH visits to their main OP in the self-reported data exceeded the total ntiniher of visits  to  all 01’s thei’ saw/or any reason in the past 12 months in their  98  administrative records. From the data presented it cannot be confirmed that these findings were due entirely. or even in part, to respondent over-reporting.  It is possible that the administrative data did not capture all hilling recorciv o/UP visits’ that were reported by respondents. For example. the respondents were asked to report the number of times the’ saw or talked on the telephone with their main UP. but the administrative data  would  not  capture telephone conversations (for which GPs are not generally compensated). Telephone calls are far less common than office visits, so the absence of information on telephone communication in the administrative records would not seem to be a plausible explanation for the large overall observed proportion of High Reporters.  Some GPs may have been paid under Alternative Payment Plan (APP) arrangements which would not have generated fee-for-service reimbursement records. APP payments are much less common than fee-for-service payments. hut the proportion of APP payments varies across different geographic areas, settings, and specialties, and so the specific number of visits that may have been provided under APP arrangements to the patients in this sample cannot he estimaled using the administrative data (Personal communication with Sandra Peterson, C l-ISPR. .Iune 20. 2006). As the individuals who were included in the analysis had at least one MH visit with a UP that was paid on a fee-for-service basis, it does not seem likely that alternative payments would have accounted for the large proportion of High Reporters.  Another possible explanation for High Reporting is that it reflects dijjirences inpatient and physician perceptions of care. This could possibly occur if a patient were to judge that a visit was associated with a MH issue, but the physician did not recognize or record it as such. ‘[he UP  may not have felt that the emotional issue was relevant to the presenting condition; they may not have had time to explore it during each visit; or they may’ not have recognized or acknowledged it initially because of other reasons (lack of familiarity with a patient; taking a ‘wait and seefl approach; uncertainty about how to address it, etc.). In such cases, the physician may not record a MR diagnostic code, hut the patient may report in a survey that they discussed their emotional. mental health or substance use problems with their UP. and therefore had a M[i visit. High Reporting was not the only type of discordance that was observed in the present study. Low Reporting was also observed, and could also have resulted from difii’rig perceptions’ by 99  physicians and patients. or  under-reporting  by’ survey respondents. or an  over-reprL’SeflfaliOfl  of  M I-I visits in the administrative data. These possible factors are described below.  There are scenarios in which a GP may judge that MR issues pertained to some visits, and a patient may not perceive it in the same way. A diagnostic code in the administrative record reflects the circumstances at the time of the visit. As more information becomes available, the GP’s assessment of the diagnosis may change. For example, a patient with a mental disorder may present with physical symptoms. and the OP may first want to rule out an emotional cause for the symptoms. Even if the OP later attributed the symptoms to a physical illness, the MR diagnostic code would remain in the administrative record for the previous visit. It is reasonable to expect that such a visit may not be counted as MR care by a survey respondent providing a retrospective self-report.  Low Reporting could also occur if some individuals in this sample failed to recall all of their MR visits over the past 12 months. As described in Section 1.3.1 of this thesis, a general tendency towards Low Reporting has been observed in comparisons between self-reported any  OP  visits (for  reason but not specifically for MI-I reasons) versus provincial administrative data’ L  12  In  those studies, it was unlikely that Low Reporting would have been due to an over-representation of OP visits in the administrative data because there are checks and balances in place to minimize over-hilling by GPs. Therefore, those findings were suggestive of under-reporting h respondents.  In the present study, which examined GP visits pertaining specifically to MR diagnoses, it is possible that some cases of Low Reporting could have resulted from an over-representation 0f AJJJ diagnosuc codes in the administrative data. Steele  et  alIô estimated that, for approximately  15% of administrative records with a MI-I code in a sample from a family practice clinic in  Ontario, no MN diagnosis was recorded in the corresponding medical chart (however, the reasons for such discordance are unknown, and it is unknown if emotional issues were discussed during those visits but not recorded in the medical chart). The causes of Low Reporting in the present sample cannot be concluded from the analyses that were conducted for this study. In  summary, it appears that either (or both) under-reporting by respondents, or an over-estimation of MN visits in the administrative data could contribute to Low Reporting, and both possibilities should be explored in future research. 100  A larger sample would permit quantitative analyses of demographic and MH characteristics associated with disagreement pertaining to the number of MH visits. Such information would help researchers who collect, analyze. or interpret these types of data to understand more about how, and for whom, the two data sources may differ, with a view to ameliorating or adjusting for such differences. Qualitative research would be useful to assess the possibility that individuals misreported the number of MH visits to their main GP, and to examine potential causes ol’ misreporting such as confusion about what was being asked by the survey question; difficulty  recalling which GP visits in the past year pertained to MR issues; methods of counting or estimating the number of MI--I visits; classifying other health professionals as a family doctor; or confusion about which visits occurred within the study reference period. Qualitative research would also be useful in order to obtain information about diagnostic coding of MI-I issues by GPs and medical office staff, in order to understand if and why an over- or under-representation of MH visits may occur in the administrative data for certain types of Ml I issues or certain types of patients. In addition, information about physician characteristics could be linked to administrative data for a larger sample in order to examine diagnostic coding patterns among GPs, and this may help to generate hypotheses as to whether or not there may be physician-level factors that could contribute to discordance between the two data sources. Factors associated with the use (or non-use) of MH diagnostic codes by GPs for administrative purposes would be a valuable priority for future research, particularly in light of plans to use provincial administrative records for surveillance of mental illness in the population. Meanwhile. until these issues are clarified, researchers or policymakers need guidance about which data source to use. An incidental finding of this study was that 31 .7% (raw data, not shown) of individuals in the sub-sample appeared to have seen more than one GP for MI-I issues during the 12 month reference period, according to their administrative records. This suggests that, for studies pertaining to the cost of MI-I care, the CCHS 1.2 data may be limited because it only collected information about the number of MH visits to each individual’s main GP, and not the total number of visits to any GP. In addition, while self-reported data may include more visits per primary MI—I care user, they capture fewer primary MI-I care users and this could  distort population estimates.  101  The results of Scenarios I and 2 suggest that, when using administrative data to estimate the number of MI-I visits to a GP, it may be prudent to use a range that is hounded at the upper end by the total number of GP visits (i.e. including all types of diagnoses), if the consequences of  underestimating the number  of  Mit visits could outweigh the consequences of overestimating  them. (It should be noted that the  raw  data were examined to assess whether the inclusion of  Mu visits to other GPs (in addition to the main GP) in the administrative data would reduce High Reporting, but it only improved ‘possible” agreement in 9 cases).  When conducting individual-level analyses,  such  as counting the number of visits to assess  treatment adequacy. it is important to recognize that the direction and magnitude of differences between the two data sources  are  not the  to identify what personal characteristics  same may  for all individuals. More study is needed in order  be associated with Low Reporting and High  Reporting in order to assess which data source is most appropriate for specific research questions. Meanwhile, until these factors are clarified, researchers should be aware of the potential ft)r bias in either direction.  At first glance, it may seem appealing to blend data from the two data sources in order to obtain an estimate of the number of visits for individuals who have 0 visits in one data source hut at least one visit in the other data source. In other words. if an individual had 0 visits in the sclf reported data but 4  visits  in the administrative data, a researcher may be tempted to fill the  gaps  in the self-reported data with information from the administrative data (or vice versa). However,  it may not be prudent to combine data for one variable from two data sources that can yield different estimates. Apparent diffi.rences between sub-groups may be obscured or artificially inflated by noise introduced from combining data from variable. Parallel analyses,  using  two  different sources for the same  data from each source independently, would provide the  researcher with information about the sensitivity of the results to the data source selected. l)ata linkages could enable researchers to conduct such sensitivity analyses on individuals within a sample, and could also capitalize on the strengths of both data sources (such as the wide range of demographic and health status variables provided in survey data, and the longitudinal perspectives provided by administrative data). This study made several contributions to the literature. The study specifically examined MI-I care from GPs, who are the main source of MF-l care for most Canadians, and who are the lcus 102  of considerable research and policy attention 17  18  It used diagnostic codes to identify primary  MH care in the administrative data, and it also examined agreement using an expanded delinition that included “possible” Mi-I visits. In addition, it was the first study to compare the number of  self-reported primary MH care visits from Canada’s only national MI-I survey to provincial administrative data, thereby reflecting the actual survey questionnaire and data collection methods. The comparisons were conducted on an individual-level rather than an aggregate level. As described in Table 4.1, at an aggregate level, the two data sources yielded almost the same total number of visits in this sample, despite the fact that the prevalence of primary Mi-I care was twice as high in the administrative data than the self-reported data. Thus, an aggregate-level comparison alone would have obscured individual-level differences between the two data sources.  This study focussed on primary MH care for reasons that have been described previously. Visits to other types of physicians could be examined in future research, ideally with a larger sample. It is unknown how individuals may classify and count contacts with physicians during overnight hospital stays, when asked specifically to count the number of times they talked to their main (iP or family doctor in the past 12 months. Previous research excluded individuals with in-patient . Eight of the 145 individuals in this sub-sample had an overnight hospital stay with a 3 stays principal diagnosis of a MH condition, according to the HAS data. All of the analyses for this study were repeated without these 8 cases. to assess how their absence affected the study results. Their removal had minimal effects on the results, so all cases were retained in the l)reSenled results.  The results of this study should be considered with some caveats in mind. The sample onl included individuals who agreed to have their records linked to provincial administrative data and who provided an accurate PHN (n  2,372). Although the characteristics of the individuals  in the total study sample were similar to the characteristics of individuals in the complete CCFIS 1 .2 Public Use Microdata File for British Columbia (n = 3.902), there may he unmeasured psychological characteristics of the non-linkers that could also affect their recall and reporting of Mi-I visits, and the generalizability of the findings to the population. The main comparisons in this study were conducted on a sub-sample of individuals who had at least one visit in both data sources (Group A). This permitted non-zero estimates from one data 1 i  -,  source to be compared with non-zero estimates from the other data source. Within the sub sample of 145 individuals. 56.6% had a measured mental disorder in the past 12 months according to the CCHS I .2/WMH-CIDI. This was higher than the proportion (1 1 .4%) in the lull linked sample. These individuals appear to have had a higher number of visits than those who had evidence of primary MN care in only one data source. As reported in Table 4.1. the mean number of MI-I visits to the main GP among individuals who had at least one visit in ho/h data sources was 5.4 in the self-reported data and 3.4 in the administrative data. The mean number of self-reported MH visits to the main GP among individuals in Group C appeared lower (3.5 visits, raw data, not shown); as did the mean number of visits for individuals who had at least one visit in the administrative data but 0 visits in the self-reported data (Group B) (1 .8 visits). The higher number of visits among individuals in Group A versus the other two groups seems reasonable as the individuals in Group A appeared to have been more ill.  4.4 CONCLUSIONS  The present study demonstrated that individuals in this sample tended to report more Mi! vlsi/s  to their main GP than were found in their administrative records. The findings of this study are consistent with previous research which found that individuals reported more MH care visits than were found in their administrative records. However, further research would be needed in order to determine whether or not this is due to over-reporting by individuals, or an underrepresentation of MFI visits in administrative data (perhaps due to diagnostic coding issues or an absence of information about visits that are not captured under the fee-for-service payment system). In any case, the results suggest that the two data sources are not interchangeable. When considered with the results of Chapter 2, which found that the number ofMI-I care users was higher in the administrative data than in the self-reported data, the finding fioni this chapter indicate that the predominant direction of disagreement between the two data sources differs depending on whether any primary MH care is measured or the numher Qf visits are measured. Researchers need to be cognizant of this when selecting a data source, and they may benefit ftom conducting parallel analyses with the two data sources in order to gel a more complete picture of possible MR care use,  104  When selecting an appropriate source of data on MI-I care, researchers must consider that neither self-reported data nor administrative data are considered to he gold standard’ measures of health service use. Among the strengths of surveys is that the data are widely accessible to researchers, and the data are standardized nationally. In addition, survey data include a wide range of variables that can be used to study factors associated with the intensity of MH  service  use. However, national M H surveys are conducted very rarely, while administrative health data is available for any time period and can be used for longitudinal research. Administrative data can also provide population-level estimates that are not dependent on sampling or survey response rates. As shown in this study, both data sources have advantages and disadvantages and each should be assessed in light of the specific research question that is being examined. Even if improvements to either or both data sources were to increase the comparability of their  estimates of primary MH care use, they may each still provide different perspectives (from consumers and providers), and data linkages would enable researchers to capitalize on the strengths of each data source and would facilitate sensitivity analyses. At present. the CCI—lS only provides information about the number of visits to each individual’s main GP, so this limits its applicability for certain types of analyses. An incidental finding of this study was that approximately one-third of individuals who were included in the sub-sample  for the principal analysis saw more than one GP in the past year, according to the administrative data. This use of multiple GPs has implications for the design of survey questions about MlJ care use, and also for health services research pertaining to continuity of MH care.  1 05  4.5 REFERENCES  1. Starkes JM, Poulin CC. Kisely SR. Unmet need for the treatment of depression in Atlantic Canada. (‘an .1 Psychiatrv. 2005 ;50:5 80-590.  2. Katz Si, Kessler RC. Lin E. Wells KB. Medication management of depression in the United States and Ontario. Journal o/6eneral Internal Medicine. 1998;13:77-85. 3. Rhodes AE, Lin E, Mustard CA. Self-reported use of mental health services versus administrative records: Should we care? ml J Methods Psychialr Res. 2002;1 1:125-133. 4. Mandcrscheid RW, Rae DS, Narrow WE, Locke BZ, Regier DA. Congruence of service utilization estimates from the Epidemiologic Catchment Area project and other sources. Arch Gen Psychiatiy. 1 99350: 108-114. 5. Knox SA, Britt H. The contribution of demographic and morbidity factors to self-reported visit frequency of patients: A cross-sectional study of general practice patients in Australia. BA/(  Family Practice. 2005;5: 17. 6. Watson DE, Heppner P. Roos NP. Reid RJ, Katz A. Population-based use olmental health services and patterns of delivery among family physicians, 1 992 to 2001. Can .1 Psychiatry. 2005 ;50:398-406.  7. Bilsker 0. Goldner EM, Jones W. Health service patterns indicate potential benefit oF supported self-management for depression in primary care. Can JPsychiulry. 2007;52:86-95. 8. Petrou S. Murray L. Cooper P, Davidson LL. The accuracy’ of self-reported heallhcare resource utilization in health economic studies. In! .1 TechnolAssess Health Care. 2002A8:705710. 106  9. Roberts RO, Bergstralh Li, Schmidt L, Jacobsen Si. Comparison of self-reported and medical record health care utilization measures. J Clin Epidemiol. 1 996;49:989-995. 10. Landis JR. Koch GG. The measurement of observer agreement for categorical data. Biomeirics.  I 977;33:1 59-174.  11. Raina P, Torrance-Rynard V. Wong M, Woodward C. Agreement between self-reported and routinely collected health-care utilization data among seniors. Health Serv Res. 2002;37:751774.  12. Mustard CA. Goel V. Barer ML, Muhajarine N. Derksen S. Validity 0/self-reported of thzaizon of insured health care services. Institute/hr Work & Health Working Paper ilO8, Toronto: Institute for Work & Health; 2000. 13. Kessler RC, lJstun TB. The World Mental Health (WMH) survey initiative version of the World 1-leahh Organization (WHO) Composite International Diagnostic Interview ((‘11)1). In!,! Methods Psvchiatr Res. 2004:13:93-121.  14. Gravel R, Beland Y. The Canadian Community Health Survey: Mental [lealth and Well being. Can .1 Ps)’chia,’rV. 200550:573-579. 15. Craven MA, Cohen M, Campbell D, Williams .1, Kates N. Mental health practices olOntario family physicians: A study using qualitative methodology. Can •J Psychiairy. 1 997;42:943-949.  16. Steele LS. Glazier RH. Lin E. Evans M. Using administrative data to measure ambulatory mental health service provision in primary care. Med Care. 200442:960-965. 17. Michalak EE, Goldner EM, Jones W, Oetter HM, Lam RW. The management of depression in primary care: Current state and a new team approach. B(’MJ. 2002;44:408-41 1. 107  I 8. Wang J, Langille DB, Patten SB, Mental health services received by depressed persons who visited general practitioners and family doctors. Psychialr Serv. 2003 ;54:878-883.  108  5 CONCLUSION  This chapter will summarize the findings from the three manuscript-based chapters, highlight the unique contributions of the dissertation, and outline key implications and recommendations for researchers, policymakers, survey designers and administrative data analysts who collect, analyze and interpret MH care data.  5.1 SUMMARY OF RESEARCH FINDINGS  This dissertation directly compared self-reported data and administrative records on a person-byperson basis in a sample of 2,378 residents of the province of British Columbia. The first research manuscript, which was presented in Chapter 2, reported that the proportion of individuals in the study sample who had primary MI-I care according to the administrative data (19.3%) was approximately twice as high as the proportion who had self-reported primary MI-I care (8.5%), during the 12 months preceding their survey interview. As described in Chapter 2, similar differences have been observed between independent aggregate-level estimates based on survey data and independent estimates based on provincial administrative data. The present study strengthened evidence that such differences are not ecological artifacts, because this study directly compared the proportion of primary MH care users in self-reported data versus provincial administrative data over exactly the same 12 month reference period within one sample.  A previous study, by Rhodes ci a!, found approximately equal proportions of MH services users in self-reported data and provincial administrative data in Ontario. The present study used  diagnostic codes to identify primary MT-I care in the administrative data of British Columbia, and this method is known to detect approximately twice as many MH care users than the method that was used by Rhodes et a! 2 Based on the evidence to date, it appears that provincial ’ 1 administrative records (containing MH diagnostic codes) capture substantially niore primary MI-I care users than self-reports.  Researchers are not only interested in whether or not the two data sources provide similar prevalence estimates of primary MFI care use, but also whether or not they identify the same 109  individuals as primary MN care users. The administrative data captured most (74.4%)  individuals in this sample who reported having primary MH care; however, the self-reported data captured only one-third of the individuals who had an administrative record of care. Often researchers are interested in the use of MN care by individuals with specific characteristics, like depression, as described in the Andersen model in Chapter 1. For example, such information may be useful for assessing met and unmet needs for care. The analyses in Chapter 3 demonstrated that individuals in the sample who had a Major Depressive Episode (M[)E) in the past 12 months were more likely than those who did not, to have had primary MEl care in both data sources (Positive Agreement) than in only one data source (Disagreement). Although the two data sources were not entirely interchangeable for identifying depressed primary MI-I care users in this sample, it was encouraging that the administrative data captured almost all (92.3%) of the depressed individuals in the sample who reported having primary MR care, and the selfreported data captured most (74.1%) of those with administrative records of primary MI-I care. Although the present study cannot be compared directly to a previous study by Rhodes el a!, due to different methodologies, Rhodes ci al also found higher levels of agreement in depressed individuals than non-depressed individuals’.  In the present study, both Disagreement groups were dominated by non-depressed individuals. (Non-depressed individuals comprised approximately 90% or more of each of the Disagreement groups). By dichotomizing Disagreement into two categories, it was possible to illuminate similarities and differences between the two groups. Although there was little difference in the proportions of depressed and non-depressed individuals in the two Disagreement groups, the  Disagreement Admin record only) group was approximately six times larger than the Disagreement (Sel/Lreport only) group, and most of the individuals in the Disagreement (Admin record only) group did not meet the criteria for any of the common measured mental disorders that were assessed in the CCHS 1.2. Thus, this study revealed a very large group of primary Mil care consumers who do not appear in self-reported data, and about whose needs for care very little is known.  The analyses in Chapter 3 also revealed a large proportion (25.9%) of depressed individuals who had no evidence of primary MI-I care in either data source (Negative Agreement). ‘This suggests that at least some non-use of care by depressed individuals that has been observed in the literature cannot solely be attributed to gaps in one data source versus another. 110  Chapter 4 described the difference between the number of MH visits (to the GP seen most often) in the self-reported data versus the administrative data. Among individuals who had at least one visit to a GP in both data sources (n  =  145), the number of self-reported visits exceeded the  number of visits in the administrative data (High Reporting) in half the cases. Exact agreement was observed in approximately one-quarter of the cases, and Low Reporting occurred in the remaining cases. A second analysis was described in Chapter 4 in which the definition of exact agreement (Exact Matches) was expanded. The rationale for the expanded definition was that patients may present with both physical and MH issues but a GP can only record one diagnostic code per visit. Under this second scenario, a “possible” Exact Match occurred if the number of self-reported visits was at least as high as the number of MR visits to the individual’s main GP in the administrative data, and did not exceed the total number of visits (for all diagnoses) to the individual’s main GP in the administrative data. Under the expanded definition, “Exact Matches” increased, comprising half of the cases in the sub-sample of 145 individuals, and high Reporting was reduced to one-quarter of cases. Even with the expanded definition, only half of the individuals, at best, in this sub-sample could have had exact agreement. In some cases, a small difference in the number of visits in each data source may not seem important, depending on a particular research objective. However, as described in Table 4.2, the mean magnitude of High Reporting was not trivial (5.7 visits).  From the data presented it could not be confirmed that the findings of High Reporting were due entirely, or in part, to respondent over-reporting. Yet, it did not seem likely that such a large proportion of cases of High Reporting would be completely accounted for by an absence of MN diagnostic codes in the administrative records. While it is possible that some GP visits were captured under APP arrangements, this could not be assessed from the available data for this sample.  The results of other studies have suggested a tendency towards High Reporting of Mu visits 3 ’ 1 However, studies that have compared the number of GP visits (for y reason) in self-reported data versus provincial administrative data have found a tendency toward Low Reporiing ’ This 4 .  study added to this body of research by providing further evidence that an overall tendency toward Low Reporting of GP visits may not extend to MR visits. A study by Mustard ci a! (reviewed in Chapter 1), which encompassed four provinces including British Columbia, found ill  that provincial administrative data and self-reported data yielded approximately equal estimates of the proportion of primary care isers (for any diagnosis), and the self-reported data yielded fewer GP visits (for any diagnosis) than the administrative data. In other words, it appeared that  individuals generally recalled that they saw a GP at least once in the past year, but may have failed to count all visits. In contrast, the results of the present study seem paradoxical. In this study, the self-reported data captured /Cwer primary MH care users than administrative data, hut individuals tended to report more MH visits per person than were found in their administrative records. This paradox cannot be resolved without further research. Neither data source is considered to be a gold standard, so comparisons to a third source of information (such as medical charts, or patient-physician observations or debriefings) would be needed.  In the preceding chapters, possible explanations for the differences in estimates obtained from the self-reported data versus the administrative data were ventured, including respondent over/under-reporting of their use of primary MR care, an over/under-representation of primary MR care use according to the administrative data, or differences in patient and physician perceptions of the content of medical visits. However, the reasons for differences between the two data sources cannot be confirmed from the data presented in this thesis. Fortunately, the diagnostic code definition of MI-I care was consistent with the scope of the definition proposed by the Public Health Agency of Canada , and the self-reported data came from the CCHS, thus 6 minimizing the occurrence of discordance due to idiosyncrasies in the study definitions of primary MR care.  Summary of key findings:  •  In terms of the prevalence of primary MH care use, the findings of the present study  support  growing evidence that administrative data yield a substantially higher number of primary MFI care users than self-reported data. •  •  in terms of agreement between the two data sources, three-quarters of individuals in this sample had primary MR care in the administrative data but not the self-reported data, while one-third had primary MH care in the self-reported data hut not the administrative data. Most depressed individuals in this sample who had evidence of primary MI-I care in at least one data source were captured in both data sources (69.8%). Most non-depressed individuals  112  the sample with evidence of primary MH care were captured only in the administrative data (67.8%). •  The number of self-reported MI-I visits per person (to the GP they saw most often) exceeded the corresponding number of visits per person in the administrative data for half the individuals in the study sub-sample. When the administrative definition of MI-I visits to the main GP was expanded to include all visits to that GP, the proportion of High Reporters in the study sub-sample was reduced (but not completely).  •  Differences between the two data sources were not trivial. When examining the number of visits above or below a particular threshold, for example “4 or more visits,” the two data sources may give different perspectives. In the study sub-sample, 46.3% of individuals had four or more MH visits to their main GP in the self-reported data, versus 20.7% in the administrative data.  5.2 IMPLICATIONS AND GUIDANCE FOR RESEARCHERS WHO USE PRIMARY MENTAL HEALTH CARE DATA  The findings of this thesis have several implications for researchers and policymakers who use and interpret primary MH care data:  •  According to administrative data, access to primary MR care appears to be better than selfreported data suggest. The corollary is that GPs appear to be encountering far more individuals with MI-I issues than self-reported data indicate. This suggests, for example, that administrative data could be an important data source for conveying the need for service  delivery models that provide GPs with appropriate support for the management of MR issues. Such models may include shared care with MH specialists, increased availability of cost-effective community services, and promotion of self-managed care programs ’ 7  .  As  another example, if the consequences of overlooking unmet needs are a concern, a conservative approach would be to use survey data rather than administrative data to describe primary MR care. Survey data would also provide information from a consumer perspective about perceived barriers to care, satisfaction with care, and perceived reasons for stopping care.  113  •  While administrative data captured more primary MH care users than self-reported data, most of the individuals who were captured only in the administrative data had no evidence of any of the mental disorders that were measured in the CCHS. Although those individuals may have had other types of mental disorders that were not measured in the survey, this finding may indicate that self-reported data understate the prevalence of primary MH care use by individuals without disorders. Alternatively, it is possible that the GPs were reporting valuable uses of MH care for prevention, evaluation and emotional support that may not have been recognized, recalled, or reported as MH care by survey respondents. Nonetheless, in an era of limited resources, it is important to ensure that primary MFI care is optimally allocated, and more study of this group of primary MR care users is warranted. Linkages between selfreported data and administrative data could provide more information about the characteristics of individuals who access care, and those who do not.  •  It is important to recognize that the direction and magnitude of differences between the data sources were not the same for all individuals in this sample. More study is needed in order to identify what personal characteristics may be associated with Low Reporting and High Reporting in order to assess which data source is most appropriate for specific research questions. Meanwhile, until these factors are clarified, researchers should he aware of the potential for bias in either direction. Researchers should be encouraged to consider conducting parallel analyses, using each type of data, to inform questions on the use of primary MH care. This can be facilitated by using linked data sets.  •  Evidence of use of care does not imply that effective treatment was provided. Linking survey data to administrative data could provide information about outcomes from consumers’ perspectives. Data linkages between surveys and additional administrative data sets would also be valuable. For example, linkages with PharmaNet, which provides information about all filled prescriptions in British Columbia, could enhance current understanding of MR treatment practices in primary care.  •  When administrative data are used to count the number of MR visits to GPs, it may be useful to use the total number of GP visits (for any diagnosis) as an upper bound for sensitivity analyses, in case some MR visits were under-represented in the administrative data because MH diagnostic codes were not recorded for every visit in which MH issues were discussed. 114  Even then, some GP visits may be missing if they were provided under APP arrangements. For this study, this proportion was not likely high because alternative payments accounted for approximately 10% of medical services in British Columbia overall during the study period and spanned a range of services and specialties beyond primary care. 1-lowever, growth in APP payments could results in a loss of information if some form of shadow billing (or other electronic record-keeping system) is not implemented.  •  An incidental finding of this study was that approximately one-third of individuals in the study sub-sample in Chapter 4 appeared to see more than one GP about their MI-I during the 12 month reference period. This suggests that, for studies pertaining to the total number of primary MR care visits (such as economic studies), the CCHS 1.2 may be limited because it only provides information about the number of MH visits to each individual’s main GP, and not other GPs.  •  In addition to differences in the estimates provided by each data source, researchers also need to consider the strengths and weaknesses of each data source, some of which are described below:  o  Currently, survey data are more readily and rapidly accessible for individual researchers than provincial administrative data, and methodological issues tend to be straightforward. Aspects of data collection and processing that could affect the use and interpretation of survey data are generally well-documented by Statistics Canada in user manuals. Most facets of data preparation and primary variable construction for national survey data sets are conducted by Statistics Canada, and not by the end users of the data. Thus, the results of survey-based research are generally quite interpretable by a wide audience of readers because of the clarity, and national standardization, of variable definitions. Self-reported data can also provide detailed information pertaining to potential correlates of care such as demographic details, measured disorders, comorbidity, symptoms, self-perceived need for care, perceptions of care, and so on. However, population MH surveys are conducted rarely, and they rely on the willingness and ability of the respondents to accurately report the kind information that a researcher is seeking.  115  o  Administrative data can be used to examine populations without the need for sampling. They can provide on-going longitudinal perspectives. Analyses are not bound by the dates of survey interviews, or survey reference periods. However, administrative data only capture information about individuals who have been in  contact with the medical system, and so they are not suitable for studying unmet needs. In addition, in comparison to survey-based research, the esoteric nature of administrative data can result in a myriad of methodological approaches. Administrative data are primarily designed for financial management purposes and not for research purposes. At this time, data preparation and variable construction for MI—I services research is largely conducted on raw administrative data sets on an ad  hoc basis for individual research projects. The use of administrative data for the proposed national MH surveillance program for treated mental disorders may standardize variable definitions and expand the use of provincial administrative data for MH services research. As noted previously, administrative data are not medical chart data, and some gaps may occur when services are provided through alternative payment arrangements or if MH diagnostic codes are not recorded for every visit in which MH issues were discussed.  5.3 CONTRIBUTIONS  The strengths and limitations of each study were described in the preceding chapters. This dissertation as a whole made several unique contributions to the expansion of knowledge. It was the first study to compare self-reported primary MU care data from Canada’s first national MH survey to provincial administrative data (using the diagnostic code definition of primary MH care). The CCHS is currently the only national source of detailed data on primary MI-I care, and is widely used, so this enhances the present study’s relevance to researchers and survey methodologists. In addition, this research is optimally timed to inform the design and testing of the next CCJ1S on mental health. It was the first agreement study of this kind to use the diagnostic code definition of primary MI-I care to identify MH visits in the provincial administrative data, and it is optimally timed to inform the interpretation of data from Canada’s new national MH surveillance program. 116  It was also the first study to compare self-reported primary MH care and provincial administrative records in the province of British Columbia. The creation of an analytic database to conduct such comparisons using individual-level linkages between data files was described in detail in Chapter 1. independent aggregate estimates from CCHS data, and from administrative data in other provinces (where available), suggest that the overall direction of difference between prevalence estimates in the two types of data sources may be generalizable to other such as Ontario and Manitoba ’ 9  tO  provinces.  Further study would be needed to confirm this.  The methodology used to compare the two data sources enhanced the existing literature. Four agreement categories, rather than two, were examined in Chapters 2 and 3. The expansion ola definition of a MH visits was explored in Chapter 4. Moreover, attention was given to presenting data that could help a variety of audiences assess both data sources for specific research purposes. This included the use of multiple measures of agreement, the presentation of data in contingency tables, and so on.  ‘To ensure that the broad conceptual framework was appropriate, the author obtained approval to go to Statistics Canada offices and field sites to review the questionnaire design and testing processes; to observe interviewer training and survey interviews; to debrief survey interviewers about how individuals respond to questions about visits to physicians; to review infbrmation about post-interview data processing procedures and to examine other documentation. Information pertaining to the collection, processing, auditing, analysis and interpretation of provincial administrative data was obtained from discussions with staff from the Ministry of  Flealth, CHSPR, CARMHA; documentation from the Ministry and CHSPR; anecdotal reports from numerous GPs and their office staff; and through an arrangement to observe a demonstration of the electronic billing process at an academic family practice clinic in Vancouver.  Knowledge transfer for this study is uniquely facilitated because there are five organizations in Canada for whom this research has direct relevance, and to whom this research can be directly and immediately disseminated. These include Statistics Canada and the Public Health Agency of Canada (PHAC). The author has had communication with both agencies during this research process. The results of this study have also been presented to staff at CHSPR who are tasked 117  with distributing information and analyzing provincial administrative data in British Columbia. In addition, the findings of this research project have been presented to the Canadian Academy of Psychiatric Epidemiologists. CAPE is comprised of researchers from virtually all rnjor psychiatric epidemiology research programs in Canada, and some of its members have been advisors for the development of the CCHS and have participated in PHAC’s national MI-I surveillance feasibility study. Further, one of the members of the academic supervisory committee for this thesis is the Chair of the Advisory Committee on Science for the newly formed Canadian Mental Health Commission. Thus, the results of this thesis can he disseminated swiftly to decision-makers through these channels, and the findings will also he communicated more widely through scholarly literature.  5.4 FUTURE RESEARCH  Specific information needs that were illuminated in the present study are described below. These information needs are first described in the context of the process of translating primary MI-I service use into administrative data, which was described in broad strokes in Figure 1.3:  First, an encounter must generate a billing record. The administrative (MSP) database captures GP services that were reimbursed under the provincial fee-for-service funding system. As noted previously, details of encounters which occurred under Alternative Payment Plan (APP) arrangements are largely unknown to the general research community in British Columbia who use and interpret studies based on administrative data. APP arrangements represent a growing proportion of provincial health services, so it is important to consider ways that the researchers and government partners can address this information gap. For example:  •  Implementation of standardized and audited shadow billing systems for APP services would enhance health services research by facilitating more comprehensive tracking of temporal changes in service patterns, and estimations of treatment prevalence of mental disorders.  •  Meanwhile, the timely documentation and dissemination of detailed information about the types and locations of services that are funded under APP arrangements would help researchers to assess potential biases resulting from missing data.  118  Second, the identification of primary MR care in the administrative data depends on the diagnostic codes submitted by physicians. As noted previously, very little is known about how physicians and their staff choose which diagnostic code to submit with a billing record, and to what extent hierarchical statistical relationships could exist between medical office characteristics, physician characteristics, patient characteristics, and diagnostic codes; and how this could affect research pertaining to primary MH care. It would be beneficial for future research to explore and address these lacunae through, for example:  •  Quantitative analyses using linkages of existing data sets pertaining to physician characteristics, practice patterns, and patient-level diagnostic information;  •  Qualitative explorations of the context of medical encounters in relation to diagnostic coding practices;  •  The dissemination of these and related research findings through a centralized documentation system or bibliographical database (via CHPSR or the Ministry of l-lealth) to enable all analysts who use administrative data to locate and access such resources readily.  Information needs pertaining to the enhancement of survey data were also identified, for consideration by cognitive scientists and survey methodologists. These are described in the context of the process of translating primary MFI service use into survey data, which was depicted broadly in Figure 1.3. Statistics Canada incorporates techniques to maximize data quality in the design, testing, and implementation of their surveys. The consultation process for the next CCHS on mental health is now beginning, so several issues could be considered:  •  The analyses in this dissertation offered a rare opportunity to examine a specific national survey question carefully and to compare it to another data source. In light of the fact that only 50% of individuals in this sample who had primary MH care in their administrati\e records reported care in the survey, it would be valuable to examine this question closely using qualitative interviewing techniques. It should be a priority to explore how individuals interpret the question and how well they understand it. In addition, it would be useful to assess whether there are factors that may affect respondent recall or reporting of primary MH care that could be ameliorated through changes to the questionnaire, or through interviewer training or other mechanisms.  I 19  •  In addition, qualitative interviewing techniques and interviewer debriefings may he useful to illuminate factors that could cause respondents to report more (or fewer) MI-I visits than they had. How did they arrive at the number of MH visits? Were MI-I issues the only issues addressed during the visit, or the main issues, or secondary issues?  •  Cognitive scientists have explored various methods with which survey respondents recall and report behavioural frequencies ’ 11  12  For example, as the number of visits increases,  individuals may be inclined to construct rates based on the frequency of visits over a particular time period. It may be time-consuming and effort-intensive to construct an exact estimate, so a respondent may report an estimate that is based on a value on a numeric scale (such as a multiple of 10) or a calendar scale (such as a multiple of 12). This possible phenomenon was observed within the sample of the present study: in the self-reported data almost all individuals who had more than 6 visits reported multiples of 10 or 12 visits, while the administrative data revealed no such pattern. Quantitative research is currently being conducted on this phenomenon among CCHS respondents, using this linked data set (manuscript in progress). Qualitative research could explore whether individuals assume thai they are expected to provide an exact or approximate number, and whether probing questions could enhance accuracy. During the consultation process for the next CCHS, it would he useful to determine what level of precision is needed by researchers. In the CCHS, the question about the number of MI-I visits to a respondent’s main GP introduced a series of questions about their perceptions of the care they received from that GP. However, as noted in Chapter 4, researchers and planners are also interested in knowing about a respondent’s  total number of MH visits to GPs, not just the number of visits to one GP. The following issues could he considered during the consultation process for the content of the next CCIIS on mental health:  •  Is information about the total number of MH visits to all GPs important to researchers and policymakers?  •  Would information about the number of GPs seen, and reasons for seeing multiple GPs be of value?  Any recommendations for alterations or additions to the next CCI-IS must take into account that questions that appear in the CCHS are designed to optimize comparability with other health 120  surveys in Canada (currently and over time), and with MH surveys from other countries. Therefore, any changes must he considered within a broader context. Also, when proposing additions to the survey questionnaire, it is important to recognize that additions increase the time burden on the respondent and may have to he balanced by subtracting other questions.  In total. the suggestions that have been described in this section arise from the findings and challenges described in previous chapters, which quantified differences between the two data sources and laid the foundation for future research that could potentially strengthen the quality of MI-I care data and enhance MH services research.  5.5 CONCLUSiONS  Mental disorders impose substantial social, economic, and personal burdens on individuals, families, communities, and employers in Canada. GPs are on the front lines of the battle to reduce these burdens, through recognition, prevention and treatment. The results of this thesis indicate that GPs may encounter far more MR care users than survey data suggest. From a consumer perspective, this is an encouraging finding because it may mean that more people are seeking and obtaining care, or that GPs are recognizing MR issues more often than survey data suggest. From a provider perspective, the picture is less sanguine, and it suggests that attention must be focused on MR care delivery models that support GPs, and on targeting limited resources to those most in need. In this study sample, self-reported data and administrative data were not interchangeable for identifying primary MH care users or for identifying the number of visits to GPs. Researchers and policymakers need to be aware that these two different data sources can produce different estimates. Neither data source is a gold standard. It is important to be aware that, while administrative data provided higher estimates of the prevalence of primary MI-I care use, and self-reported data provided higher estimates of the number of MH visits per person, these general tendencies did not apply to all individuals. Moreover, this study provided evidence to suggest that agreement between the two data sources may not be randomly distributed across personal characteristics. For example, in this sample, depressed individuals were more likely than non depressed individuals to have had agreement rather than disagreement between their self reported data and provincial administrative records. 121  It is important to emphasize that use of care does not imply that effective treatment has been provided. Data linkages could provide more information about treatment and consumer perceptions of care that could guide policy and practice. Data linkages would also be beneficial in order to capitalize on the strengths of both types of data sources, such as the demographic and health status information available from surveys, and the longitudinal perspectives provided by administrative data. Parallel analyses of primary MR care, using data from each source, may help researchers to assess a research question from the perspectives offered by each, and to conduct sensitivity analyses pertaining to the estimates provided by each data source.  Nested within the conceptual framework described in Chapter 1, the findings of this thesis have provided evidence with which survey methodologists and administrative data analysts can consider reasons for differences in the way that primary MH care use is represented by each data source, with the ultimate aim of optimizing the potential strengths of both data sources for research purposes. In addition, by illuminating the possible direction and magnitude of differences between the two data sources, the results of the present study can inform researchers and policyrnakers who design and interpret studies of primary MR care, with the ultimate goal of enhancing evidence-based MH policy and practices aimed at improving the health and well being of Canadians.  122  5.6 REFERENCES  1. Rhodes AE, Lin E, Mustard CA. Self-reported use of mental health services versus administrative records: Should we care? IntJMethods Psychiatr Res. 2002;ll:125-133. 2. Lin E. Defining a mental health and addictions ‘case’ for national surveillance purposes: Ontario data. Toronto: Annual Symposium of the Canadian Academy of Psychiatric Epidemiologists; 2006. 3. Manderscheid RW, Rae DS, Narrow WE, Locke BZ, Regier DA. Congruence of service  utilization estimates from the Epidemiologic Catchment Area project and other sources. Arch Gen Psychiatry. 1993;50:l08-1 14. 4. Raina P, Torrance-Rynard V, Wong M, Woodward C. Agreement between self-reported and routinely collected health-care utilization data among seniors. J-Iealth Serv Res. 2002;37:751774. 5. Mustard CA, Goel V, Barer ML, Muhajarine N, Derksen S. Validity o/sel/reporied utilization of insured health care services. Institute/or Work & Health Working Paper # 108. Toronto: Institute for Work & Health; 2000. 6. Gilbert C. Use ofprovincial administrative data/Or surveillance of mental disorders:  Feasibility study. Halifax: Public Health Agency of Canada; 2006. 7. Michalak EE, Goldner EM, Jones W, Oetter HM, Lam RW. The management of depression in  primary care: Current state and a new team approach. BCMJ. 200244:408-4l 1.  1  I Li  8. Craven MA, Cohen M, Campbell D, Williams J, Kates N. Mental health practices of Ontario family physicians: A study using qualitative methodology. Can J Psychiatry. 1 997;42:943-949. 9. Watson DE, Heppner P, Roos NP, Reid R.J, Katz A. Population-based use of mental health services and patterns of delivery among family physicians, 1992 to 2001. Can.! Psychiatry. 2005;50:398-406. 1 0. Lin B, Goering PN. The utilization ofphysician services for menial health in Ontario. Institute for Clinical Evaluative Sciences in Ontario; 1999. Ii. Conrad FG, Brown NR, Cashman ER. Strategies for estimating behavioural frequency in survey interviews. Memory. I 998;6:339-366. 12. Blair B, Burton S. Cognitive processes used by survey respondents to answer behavioral frequency questions. Journal of Consumer Research. I 987;1 4:280. D/i  124  Appendix A  125  -  Table A.1: Distributions of demographic and mental health indicators in the complete CCHS Public Use Microdata File for British Columbia, and the linked study samplea •  Full CCHS sampleb n  %  Linked study sampl& n %  Age l5thru24 25thru34 35thru64 65thru74 75+  546 603 1,913 427 413  14.0 15.5 49.0 10.9 10.6  269 290 1,196 320 303  11.3 12.2 50.3 13.5 127  1,831 2,071  46.9 53.1  1,052 1,326  44.2  430 660 853 866 728 365  11.0 16.9 21.9  1 977 886 1,030  Sex Male Female  558  Household income Lessthan 15,000 $15,000-$29,999 $30,000-$49,999 $50,000-$79,999 $80,000ormore Missing  —  Marital statusd Married I Common Law  Widowed I Divorced I Separated Single  261  11.0  419 566  176 23.8  22.2  540  227  18.7 9.4  468 124  19.7 5.2  50.7 22.8 26.3  1,269 589 517  53A 24.8 21.7  855 710 1,848 443  21.9 18.2 47.4 11.4  534  22.6  419 1,143 263  17.8 48.5  226 3,646 30  934 0.8  116 2,250 12  513 3,214 175  13.1 82A 4.5  271 2,013 94  11.4 847  309  8.7  203  3,565  9Z0  275  8.5 9 1.5  —  Educationd  Less than secondary Secondary graduate Post-secondary graduate Other  -  Major Depressive Episode Yes  —  past 12 months  No Missing  Disorder! Substance Dependence  —  Yes No___  SAMPLE SIZE  —  5.8  4.9  94.6 0.5  past 12 months  Yes No Missing  TalkedtoGPabout MH  11.1  4.0  past 12 monthse -—  3,902  2,378  aunweighted bSource: Canadian Community Flealth Survey on Mental Health and Well-being (CCHS, cycle 1.2) Public Use Microdata File for British Columbia, 2002 (n 3,902). CSource: Linked data from a sub-sample of the CCHS (cycle 1.2) & the British Columbia Medical Services Plan (n = 2,378). dMissing cases represented less than 1% of total and are not displayed. Percentages are based on a denominator of 3,902. elndividuals for whom primary MH care data were missing were excluded from the linked study sample (n = 5) so the denominator used here to calculate the percentages for primary MI-I care in the full CCHS sample also excludes missing cases (n = 28).  126  Appendix B  127  Questions on mental health service utilization from the Canadian Community Health Survey on Mental Health and Well-being which were used to examine self-reported primary MH care Questions pertaining to the use of primary MH care in the past 12 months:  During your lifetime, have you ever seen, or talked on the telephone, to any of the following professionals about your emotions, mental health or use of alcohol or drugs? I Psychiatrist 2 Family doctor or general practitioner 3 Other medical doctor such as a cardiologist. gynaecologist or urologist 4 Psychologist 5 Nurse 6 Social worker, counsellor or psychotherapist 7 Religious or spiritual advisor such as a priest, chaplain or rabbi S Other professional 9 None You mentioned that you saw, or talked on the telephone, to a family doctor or general practitioner about your emotions, mental health or use of alcohol or drugs... When was the last time? 1 During the past month 2 Between 2 and 6 months ago 3 Between 7 and 12 months ago 4 More than a year ago The question pertaining to the number of visits to the GP seen most often in the past 12 months:  Think of the family doctor or the general practitioner you talked to the most often during the past 12 months. I-low many times did you see, or talk on the telephone, to this family doctor or general practitioner (about your problems with your emotions. mental health or use of alcohol or drugs)?  I 28  Appendix C Certificate of Approval from the Behavioural Research Ethics Board of the University of British Columbia  129  The University of British Columbia Office of Research Se,vices Behavioural Research Ethics Board Suite 102, 6190 Agronomy Road, Vancouver B. C. V6T 1Z3  CERTIFICATE OF APPROVAL- MINIMAL RISK RENEWAL UBC BREB NUMBER: DEPARTMENT: UBC/Medicine Faculty of/Health H03-80750 Mieke W. Koehoorn Care & Epidemiology INSTITUTION(S) WHERE RESEARCH WILL BE CARRIED OUT: Site Institution I Hospital) UBC (excludes Vancouver UBC  PRINCIPAL INVESTIGATOR:  Other locations where the research will be conducted:  N/A  CO-INVESTIGATOR(S): ‘oAnne L. Palm SPONSORING AGENCIES: N/A PROJECT TITLE: Bias in Self-Reported Health Services Utilization in the Canadian Community Health Survey: Influences of Personal Characteristics, Health Status and Patterns of Care EXPIRY DATE OF THIS APPROVAL: October 30, 2008 APPROVAL DATE: October 30, 2007 The Annual Renewal for Study have been reviewed and the procedures were found to be acceptable on ethical grounds for research involving human subjects. Approval is issued on behalf of the Behavioural Research Ethics Board  130  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.24.1-0071438/manifest

Comment

Related Items