Open Collections

UBC Theses and Dissertations

UBC Theses Logo

UBC Theses and Dissertations

A case study of the utility of focus groups for program evaluation involving non-English speaking program.. Ritch, Adele Denise 1997-12-31

You don't seem to have a PDF reader installed, try download the pdf

Item Metadata


ubc_1997-0595.pdf [ 3.87MB ]
JSON: 1.0078373.json
JSON-LD: 1.0078373+ld.json
RDF/XML (Pretty): 1.0078373.xml
RDF/JSON: 1.0078373+rdf.json
Turtle: 1.0078373+rdf-turtle.txt
N-Triples: 1.0078373+rdf-ntriples.txt
Original Record: 1.0078373 +original-record.json
Full Text

Full Text

A CASE STUDY OF THE UTILITY OF FOCUS GROUPS FOR PROGRAM EVALUATION INVOLVING NON-ENGLISH SPEAKING PROGRAM PARTICIPANTS by ADELE DENISE RTTCH B.Ed., The University of Alberta, 1978 A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF ARTS in THE FACULTY OF GRADUATE STUDIES Department of Educational Studies We accept this thesis as conforming to the required standard THE UNIVERSITY OF BRITISH COLUMBIA October 1997 ©Adele Denise Ritch, 1997 In presenting this thesis in partial fulfilment >of the requirements; for an advanced degree at the University of British: Columbia, I agree that the Library shall make it freely available for reference and study. I further agree that permission for extensive copying of this thesis for scholarly purposes may be granted by the head of my department or by his or her representatives. It is understood that copying or publication of this thesis for. financial gain shall not be allowed without my written permission. . >,'.•.• Department of • f~~<d[ The University of British Columbia Vancouver, Canada .V'- • '-. Date Ar-VoW/- 1 %[ I ^ ^1- •'•. DE-6 (2/88) ABSTRACT In an English language context, the ability of program participants with limited English competency to participate in program evaluation processes is restricted. However, when program participants are invited to discuss their experiences in their preferred language, they make meaningful contributions as program stakeholders. Within the context of a program evaluation of the Nobody's Perfect Parenting Program (Ritch & McLaren, 1994), a case study approach was used to determine the utility of focus groups as a program evaluation methodology with non English speaking program participants. Six focus groups were facilitated by bilingual, bicultural facilitators using a set of questions to encourage participants to discuss their experiences in the program and to offer suggestions for program improvement. Analysis of these focus groups yielded new and useful information for program planners and policy makers. This work showed that people who are generally excluded from research samples because of linguistic barriers are able to participate as stakeholders in the evaluation process when their participation is sought in their preferred language. In their own language, participants provided feedback to program planners and policy makers which they were not able to provide in English. Through this process, their response to the program and their recommendations for improvement became known. The inclusion of program participants in the evaluation of the Nobody's Perfect Program focused attention on detennining evaluation methodologies which would effectively include program participants from diverse cultural groups who are not English speakers. iii TABLE OF CONTENTS Abstract ii Acknowledgment v Chapter One: Introduction 1 mvorving Non English Speaking Participants in Program Evaluation 1 Description of the Nobody's Perfect Parenting Program Evaluation 3 Choosing Focus Groups as an Evaluation Method 6 Case Study of Focus Groups Using Languages Other Than English 8 Chapter Two: Literature Review 11 Focus Groups in Program EvaluationUnder Representation of Minority Groups in Program Evaluation 14 Using Focus Groups With "Special" Populations 20 Use of Language in Research Processes 23 Chapter Three: Methodology 28 The Case Study ApproachDeveloping the Focus Group Questions 30 Focus Group Facilitators 32 Cross Cultural Facilitation of Focus Groups 6 Focus Group Composition 8 Permission to Participate 41 Support for Parent Participation 2 Conducting the Focus Groups 3 Recording, Transcription and Translation of Focus Group Data 44 Chapter Four: Analysis of Focus Groups 48 Understanding Participants' Response to the Nobody's Perfect Program 48 Why did you Choose to Participate in Nobody's Perfect? 51 What did you Like About the Nobody's Perfect Program? 5 What did you Learn From the Nobody's Perfect Program? 58 Did the Leader of the Nobody's Perfect Program Seem More Like a Teacher or a Facilitator? 62 Did the Nobody's Perfect Program Seem Appropriate to Your Culture? 67 How Were the Nobody's Perfect Books Useful to you and What Suggestions do you Have for Improvement? 69 What Ideas do you Have for Improving the Nobody's Perfect Program? 74 iv Chapter Five: Summary and Conclusions 76 Involvement of Non English Speaking Program Participants in Program EvaluationLimitations of Focus Groups 80 Translation Processes 3 Program Evaluation Standards 4 Bibliography 85 Appendix 91 ACKNOWLEDGMENT I wish to thank the members of my research committee, Dr. Judith Ottoson and Dr. Bernie Mohan for their enthusiasm and insight. I particularly thank my Faculty Advisor, Dr. Tom Sork, for his patience, flexibility and clarity throughout this study. I also wish to thank my family and friends who in so many ways supported the completion of this thesis. I express sincere gratitude to my life partner, Dianne Liscumb. Her involvement throughout the study and critique of the many drafts enriched all aspects of this work. 1 CHAPTER ONE INTRODUCTION Involving Non English Speaking Participants in Program Evaluation The demand for accountability of publicly funded programs is on the rise creating an increasing need for relevant and comprehensive evaluation of such programs. Family support services such as parenting programs offered through community-based non-profit agencies are a particular type of publicly funded program which require the development of evaluation strategies which are effective and appropriate to the settings and circumstances typical of such services. At many community service agencies, family support programs now include programs conducted in languages other than English for families of diverse cultural and linguistic backgrounds who prefer to attend programs where their first language is spoken. Evaluation of these programs offers particular challenges for program evaluation. To date, evaluation of family support programs has been done primarily with white, middle to upper class, English speaking populations. The omission of program participants who do not speak English from evaluation studies leads to the underreporting of the impact of these programs in these cultural and linguistic communities (Johnson, Beiser & Krech, 1991; Lincoln, 1991; Madison, 1992; Morgan, 1993) . While there are numerous studies of parent education programs using various research designs, few investigations have been directed 2 toward evaluating the effectiveness of parenting programs for parents of different educational and socioeconomic status (Dangel & Polster, 1985) . As well the common practice of identifying bilingual bicultural individuals (often staff) to act as a "bridge" between program participants and the program evaluator does not address the need to integrate program participants into evaluation processes. In order to respond to the cultural and linguistic barriers to participation of program participants in the evaluation of community-based family support programs, it is crucial for evaluators to develop comprehensive, practical and effective evaluation methods that address the reality of the family diversity usually found in the general population today. The important question for evaluation of social programs is not whether the process used is capable of detennining truth (Madison, 1992) . It is more critical that the "discovery of truth entail mechanisms for input from the populations most directly affected by social policy" (Madison, 1992, p. 1) . Madison contends that evaluators have an "ethical and moral obligation to examine the efficacy of existing evaluation technologies in determining the impact of social programs on the lives of the poor and minorities" (Madison, 1992, p. 1) . Evaluators of social programs must consider evaluation methods that address the particular needs of non English speaking participants enabling them to be included as stakeholders in the evaluation process. It must be recognized that when people have limited English competency, their ability to participate in meamngfuL evaluative conversations conducted in English, about their experiences in programs is limited. Within an English language context, information 3 gathered from program participants with limited English skills will always be less complete than that of first language English speakers. However, when program participants are invited to engage in the evaluation process in their preferred language, they have an opportunity to make a meaningful contribution to the evaluation process (Guba & Lincoln, 1989) . As the number of family support programs in languages other than English increases, it becomes critical to address the complexities inherent in the evaluations of such programs through the development of methodologies which will encourage the involvement of program participants and provide useful information for program planners and policy makers. Description of the Nobody's Perfect Parenting Program Evaluation The Nobody's Perfect Parenting Program is described as being based on an "adult education model." The program manual describes this model as one where participants learn through discussion with others, rather than from an "expert" providing information. This model distinguishes Nobody's Perfect from other well-known parenting programs where a particular approach to parenting is "taught" to parents by a professional educator. Nobody's Perfect is based on the premise that parents attending the group have experience and knowledge about parenting and that they come together to learn more by sharing their experiences with other parents. The goals of the Nobody's Perfect Program are: • To increase parents' knowledge and understanding of children's health, safety and behaviour. • To effect positive change in parents' behaviour relative to their children's health, safety and behaviour. • To improve parents' coping skills. • To improve parents' confidence and self-esteem as parents. • To build self-help and mutual support networks. Groups of 8-12 parents meet once a week for 6-8 weeks to discuss parenting issues. These groups are facilitated by hidividuals trained in facilitation methods and who are familiar with the content and philosophy of the Nobody's Perfect Program. These group facilitators work with the group to develop an agenda which relates to the interests of the group participants. Nobody's Perfect program materials include five books entitled "Body," "Safety," 'Mind," "Behaviour," and "Parents." These books contain information about children's physical, intellectual and emotional development, injury prevention, and problem solving. These plain language, illustrated books are published by Health Canada and are provided to all program participants free of charge. Criteria for participation in the Nobody's Perfect Program includes parents with children under six years old who may be young, single, have low education and have low incomes or who may be socially, geographically, or culturally isolated. Program fees, lack of child care and transportation needs are recognized as common barriers preventing many parents from participating in parenting programs (Cross, 1981; Stevens, 1993) . In recognition of these obstacles, the Nobody's Perfect Program is offered at no charge and includes on-site child 5 minding Transportation costs are covered for parents who express a need for this support and snacks or meals are frequently provided. These supports are an integral part of the implementation of the Nobody's Perfect Program throughout British Columbia and are considered essential to allow parents of small children to attend. In Vancouver, many parents participate in the program in languages in addition to English including Cantonese, Punjabi, Spanish, Vietnamese, and other languages as required. Over the past ten years numerous evaluations from multiple perspectives have been done federally and provincially to determine the effectiveness of the Nobody's Perfect Program in meeting its goals and objectives. Each of these evaluations has identified Nobody's Perfect as a highly successful program (Brochu, 1992; Rivers and Assoc., 1990; VanderPlaat, 1988) . However, none of these evaluations addressed implementation of the Nobody's Perfect Program with participants who speak languages other than English particularly the various cultural and linguistic aspects of program content and delivery. Because of linguistic and cultural barriers non English speaking parents were unable to contribute to the usual program evaluation processes of surveys, impact studies or satisfaction scales. In 1994, an evaluation of the Nobody's Perfect Program in languages other than English was conducted in Vancouver, Canada (Ritch & McLaren, 1994) . In the context of this comprehensive program evaluation, six focus groups were implemented with non English speaking program participants to enable program planners and policy makers to learn about the particular experiences of non English speaking program participants. These six focus 6 groups formed the basis of this current study to determine the utility of focus groups to provide new and useful information for program planning and policy making. Health Canada was particularly interested in obtaining reactions of non English speaking program participants to the program books which are a fundamental part of the Nobody's Perfect Program and are provided free of charge to all program participants throughout the country. As well, the agencies involved in program implementation and the provincial government, as a program funder, were all interested in learning more about how this program was being implemented with diverse cultural groups. Choosing Focus Groups as an Evaluation Method One of the key contextual factors which shaped the evaluation of the Nobody's Perfect Program was the inclusion of program participants who did not speak English as key stakeholders (Shadish, Cook, & Leviton, 1991) . In particular, a primary consideration in the development of the evaluation design and choice of data collection methods was an emphasis on ensuring the involvement of participants from Nobody's Perfect groups facilitated in languages other than English. In this regard, the focus group, which typically consists of approximately 8 to 12 people, facilitated in a carefully planned discussion by a skilled facilitator, was chosen as the most effective method. Focus groups are often used in program evaluation when the goal is to obtain in depth information about a particular topic. Through focus groups evaluators hope to learn about issues that are best accessed through group conversation and discussion. In this situation the focus group format provided an opportunity 7 to obtain meaningful and comprehensive information about the Nobody's Perfect Program from group participants. Focus group questions involved them in discussions about their experiences in the program and asked them for suggestions for its improvement. Morgan (1997) describes focus groups in the following manner: Focus groups are basically group interviews...where the reliance is on interaction within the group, based on topics that are supplied by the researcher who typically takes the role of a moderator. The hallmark of focus groups is their explicit use of group interaction to produce data and insights that would be less accessible without the interaction found in a group, (p.2) Morgan's description of focus group methodology is congruent with the central aspect of the Nobody's Perfect Program where participants are especially encouraged to share information, ideas and feelings with each other and build knowledge through the exchange of various personal perspectives. The similarity between the research method and the program itself made the focus group seem a most appropriate evaluation method for obtaining information from program participants. In contrast, surveys were not considered to be as appropriate for the evaluation of the Nobody's Perfect Program because the approach did not fit well with the goals of the evaluation which emphasized people's reactions to the program rather than identification of 8 specific and measurable program effects or outcomes. Also, implementation of surveys can be difficult with populations who may not be used to such instruments and who may not be literate in English or other languages (Weiss, 1988) . Finally, translation of survey instruments raises some complex issues including questions of vaUdity in a cross cultural context (Boshier, 1991; Ervin & Bower, 1953) . Use of mdividual interviews with participants was also deemed less valuable than focus groups for this evaluation. Personal interviews with participants runs counter to the previous experience of these individuals who had come to rely on group support in the Nobody's Perfect Program to express and exchange ideas. While interviews are generally ideal for obtaining information about people's feelings and experiences, they may not be prudent in all situations. The interview process involves an individual participant in an intensive conversation with an interviewer. Agency staff revealed that many program participants were "shy" and would probably not volunteer for an individual interview. These considerations along with the high cost of translation of numerous interview transcripts were strong factors in deciding to proceed with focus groups. Case Study of Focus Groups Using Languages Other Than English This thesis examines the utility of focus groups as a program evaluation methodology through a case study approach. The case study describes the development and implementation of focus groups in languages other than English and analyzes the extent to which these focus groups enabled participants to provide new and useful information to program planners and policy makers. The purpose of this case study was to determine the utility of focus groups as a program evaluation methodology with program participants from diverse cultural and linguistic backgrounds who are not confident English speakers. In the context of this study, utility is considered to be the extent to which the use of focus groups contributes to the ability of non English speaking program participants to: • Participate as stakeholders in the program evaluation process. • Provide elaborate descriptions of their experience with the program • Offer information which is useful for program planning decisions. This study indicates that people who are generally excluded from research samples because of linguistic barriers are able to participate as stakeholders in the evaluation process when their participation is sought in their preferred language. Focus group discussions in participants' first language enable them to fully contribute to the evaluation process and provide program planners and policy makers with considerable information about participants' experience in the program as well as very clear directions for program improvement. According to Brislin (1981), non English speaking participants may be invisible in the evaluation context when their opinions are not sought in their preferred language. Through the use of focus groups, non English speaking program participants provided information that would otherwise have been unheard by program planners and policy makers. Program participants became stakeholders in the evaluation process through participation in a focus group discussion in their preferred language. Understanding the utility of focus groups for providing useful information which is otherwise not available will encourage the use of focus groups as a methodology in the program evaluation field (Morgan, 1997) . 11 CHAPTER TWO LITERATURE REVIEW Focus Groups In Program Evaluation The use of focus groups is a qualitative research approach designed to obtain in depth information, understanding and ideas about a specific area of interest (Basch, 1987; Kruegar, 1994). Focus groups are becoming increasingly popular as a program evaluation method for evaluators of adult education programs to provide insight into the attitudes, perceptions, and opinions of participants (Krueger, 1994; Morgan, 1993; Morgan, 1997) . Indeed, there has been a rapid and dramatic increase in the use of focus groups in a wide range of research situations during the past ten years (Morgan, 1997) . Originally called focused interviews, this technique became popular during the late 1940's after it was first used by Robert Merton to evaluate radio programs in the United States (Stewart & Shamdasani, 1990) . Merton and others developed the focused interview technique, modifying the procedures for their own research needs. Focus groups have since become an important tool for researchers in many fields including program evaluation, marketing, public policy, advertising and conimunications (Stewart & Shamdasani, 1990). In the field of health education, health educators had used small group process as a method for program delivery, skills training, problem solving, and organizing individuals and groups, but focus groups for research purposes were not used until the 1980s (Basch, 1987) . The usefulness of focus groups as a research tool in health education, program planning and program evaluation has been widely recognized and indeed much of the literature about focus groups has been written in the last ten years (Basch, 1987; Kreuger, 1994; Morgan, 1997) . Focus groups may be used in many ways for research and both Krueger (1994) and Morgan (1997) encourage researchers to experiment with the range of possibilities for focus groups. Morgan (1997) suggests that there is no "one right way" to do focus groups and encourages researchers to make choices that will enhance both the research process and the resultant information. Basch (1987) suggests that along with the synergism vvithin groups which can uncover important understandings, there is potential for groups to provide a secure setting for individuals to express ideas, particularly those related to sensitive areas. The group process allows comments from one participant to stimulate ideas for other participants and provides opportunities for participants' own thoughts and theories about a topic to be considered alongside other ideas derived from existing theory or prior research (Morgan & Spanish, 1984) . Mullen and Reynolds (1982) argue that approaches which are concerned with uncovering the meanings, definitions and interpretations made by the subjects of the study are more likely to accurately depict their priorities than methods which begin by preconceiving that world and its meaning. Basch (1987) states that "understanding the target group's perspective is integral to achieving the goals of health education and focus group interviews are an appropriate method for understanding and developing sensitivity toward those we serve" (p. 436). 13 Focus groups are particularly well suited to investigations which hope to obtain a wide range of information, allowing participants to explore issues and suggest improvements to programs. The focus group discussions conducted as part of the evaluation of the Nobody's Perfect Program allowed the researcher to draw on the rapport previously established among participants in the Nobody's Perfect Program and therefore begin the evaluative process with participants in a safe, familiar and comfortable environment. The focus group has the advantage of allowing people time to reflect and recall experiences (Lofland & Lofland, 1995) . Responses from individual participants can spark ideas, memories, opinions or connections for other participants (Basch, 1987; Krueger, 1994; Lofland & Lofland, 1995) . The focus group, where participants are simultaneously influencing and are influenced by others, provides a natural environment for the process of exchange and building of knowledge (Krueger, 1994) . Focus groups have become a widely used tool for gathering information and many evaluators have indicated a need for further work using focus groups in "special" populations (Krueger, 1994; Morgan, 1997) . However, this work has not been widely done or reported. Yet, the focus group seems particularly suited for obtaining information from special populations especially when program initiatives such as parenting groups are targeting populations which may not be typically involved in health education and prevention programs (Lengua et aL 1992). 14 Program evaluation processes in prevention programs with participants who speak limited English are often a source of great frustration for staff in community-based agencies. English speaking agency staff report that when they seek participants' opinions about the effectiveness of programs, participants who speak English as a second language will typically give simplistic responses compared to the comprehensive feedback received from first language English speakers. Such simplistic responses are often perceived by agency staff as reticence on the part of program participants to offer opinions about their experiences, and is often attributed to "cultural differences", in some cases leading to the conclusion that people from some cultural backgrounds are either less able or less willing to reflect on their experiences and to provide feedback. Focus groups conducted in program participants' first languages provide an opportunity to explore the relationship between participants' comfort level and confidence with the language used and participants' ability to make a meaningful contribution to the evaluative process. In addition, these focus groups highUght the process of language use within a group context. Under Representation Of Minority Groups in Program Evaluation Few studies specifically address the need to conduct research in languages other than English and there is little in the literature to guide the evaluator in working within multicultural organizations or with group participants who speak languages other than English. Krueger (1994) suggests that as the nonprofit sector reaches out to diverse audiences, sensitivity is 15 needed to adapt focus groups to build on the strengths of the target audience. This need has not yet been well addressed. Morgan (1993) states that while the existing literature about focus groups concentrates on work with relatively affluent people, there is also some discussion in the literature which alludes to the use of focus groups with "hard-to-reach" populations. However, little systematic attention has been given to these focus groups (Morgan, 1993) . While it may be recognized that a particular strength of the focus group methodology is that it can be sensitive to diverse participants, little research using focus groups has been done with participants who are not white and middle or upper class (Kreuger, 1994; Morgan, 1993). Madison (1992) points out that racial and ethnic minorities as well as the poor have often been omitted as stakeholder participants in evaluation of social service programs because of the evaluation methods commonly used by researchers. Others agree that target beneficiaries are often the least likely to have their voices heard. Rossi and Freeman (1993) suggest that in social programs the beneficiaries may be "unorganized, poorly educated, and reluctant to identify themselves" (p.407) . Lincoln and Guba (1985) assert that "all stakeholder groups can expect to receive the opportunity to provide input into an evaluation that affects it (sic)...Anything else is patently unfair and discriminatory" (p. 51) . In a later work Lincoln (1991) further describes a vision of research which addresses the identified absence of various minority groups: 16 Scientific inquiry today includes commitments first, to new and emergent relations with respondents, second, to a set of stances-professional, personal, and political-toward the uses of inquiry and toward its ability to foster action; and finally, to a vision of research that enables and promotes social justice, community, diversity, civic discourse, and caring. Lincoln argues that any discussion of standards of research today necessarily signifies a radical shift in the vision of what research is, what it is for, and who ought to have access to it. (p. 278) Lincoln (1991) places the onus of responsibility on evaluators saying that most people who evaluate social programs know very little about the minority program participants' world view and "may even instinctively keep their distance" (p. 6) . Prieto (1992) echoes this view and suggests that researchers may even be choosing not to work towards overcoming this barrier in their work. In response to the recognition that evaluators may not fully understand the experiences of those they are studying, Lincoln (1991) strongly urges evaluators to heed the following advice: Rely on stakeholders who want to speak for themselves, to participate in the radical social experiment of giving voice to those who cannot be heard, and to seeing those who have been invisible...Until we comprehend the lives that we do not lead, we will never understand how to assess what our programs are doing for the persons who are living those lives, and until we understand what they confront, we will never know 17 how to formulate humane and decent programs which directly address complex social problems, (p. 6) Kkkman-Liff and Mondragon (1991) state that the language of the interview is rarely if ever considered by researchers. In their review of 69 studies, they found that none of the studies treated the language of the interview as a variable. In their own study of the health status of Hispanics in the Southwest United States, the authors found striking differences in the health of those who were interviewed in English and those who were interviewed in Spanish. Cross tabulations for nine health status variables and access to health care measurements revealed that Hispanics who were interviewed in Spanish had lower health status and poorer access to care than Hispanics who were interviewed in English (Kirkrnan-Liff & Mondragon, 1991) . These results clearly identified language as a variable of health and emphasized the importance of conducting interviews in Spanish to obtain information about this particular group of unilingual Spanish speakers. The researchers in this study emphasized that the Hispanics interviewed in Spanish would have been omitted from the usual research sample of English speaking participants with the resultant lack of awareness of the critical differences in health status between English speaking and exclusively Spanish speaking Hispanics. The authors of this study argue that the refinement of translingual and transcultural methodologies and the use of translated instruments are critical to research and that analysis of the language of interview will promote understanding of specific populations (Kirkrnan-Liff & Mondragon, 1991). 18 Stansfield (1996) identifies another example of a situation where non English speakers were not included in the collection of research data. He describes state-wide assessment programs in the United States which provide assessments of all students at several grade levels in a variety of content areas. However, where students are English language learners, they are routinely deferred from taking this test until they become English proficient or are seniors in high school where the test is mandatory. Thus, these non English speaking students lose benefits such as feedback about their progress, appropriate remediation where necessary and can easily be overlooked and forgotten by the education system. Stansfield (1996) suggests that the solution to this problem is assessment of students in their native languages. While recognizing the difficulties in the accurate translation of the tests, the author recommends translation of the test into students' native language and content assessment of students' responses in the native language. Such innovations in testing would ensure that more comprehensive data were collected on student achievement and that all students would be included in assessments of content knowledge. Another study which focused on schools in the United States identifies the barriers to school involvement by parents who do not speak English (Epstein, 1986). The authors of this study conclude that schools will need to communicate with parents in their language of proficiency and that publications and notices must be translated for parents to ensure full understanding. Also, interpreters must be available to help parents understand the information and enable parents to communicate as an equal with their children's teachers (Epstein, 1986) . These 19 efforts must be made to provide the necessary supports in order for parents to fulfill the basic obligations to support their children's schooling. Regarding the evaluation of parenting programs, Johnston et aL (1991) state that while evaluations of parenting programs indicate that they are effective in promoting positive parent-child interactions, most parent education programs have been developed and evaluated as applied to mainstream American families or, occasionally, Canadian families. In a review of the literature, these authors found little information about programs for ethnocultural communities, and even less information about their evaluation. To obtain greater information, the researchers distributed a questionnaire to 100 agencies in Canada providing primary and tertiary care to families. Through this process, they learned that while there were some parenting programs for ethnocultural communities, no evaluations of these programs were available. The authors of this study strongly suggested further research and evaluation of the effectiveness of parenting programs for ethnocultural communities. They also suggest that with the development of programs for people of diverse cultural and linguistic backgrounds, there is a need to develop evaluation designs and methodologies to include populations who are often ignored due to language and cultural differences from the program evaluator. 20 Using Focus Groups With "Special" Populations One study which directly addressed the effectiveness of focus groups to obtain information about difficult-to-reach, high risk families, conducted by Lengua et al. (1992) , implemented six focus groups with 53 parents of children in elementary school. One goal of the study was to determine the effectiveness of focus groups in obtaining information about recruitment of families for programs related to the prevention of mental health problems and alcohol and substance abuse in children. The study concluded that focus groups were an efficient and inexpensive method of obtaining information for recruitment and retention of families, particularly information about unique needs and concerns of specific neighbourhoods (Lengua et al., 1992). The researchers emphasized the effectiveness of focus groups in enhancing the development and implementation of connnunity based, family focused interventions through addressing the needs of population sub-groups such as low income families (Lengua et al., 1992) . For example, this study identified the need for the provision of child care, transportation and a convenient location particularly for single parents and low income parents. Also, low income parents expressed a particular need for social support from the program This study concluded that focus groups were an effective method to help identify the specific beliefs and values of a specific population and thus can provide hrformation needed to improve or create programs for families. While the study does not mention the sub-group of parents who did not speak English and were, therefore, excluded for the most part from both parenting programs and from the focus groups used to obtain information about these programs, the study does suggest that focus groups are effective in obtaining information that is useful for program planning decisions. The study highlights the need to consult with various subgroups regarding the effective implementation of family programs in their communities. The study also clearly invites further research into the use of focus groups with particular subgroups with the view towards enhancing community programs. Another study which addresses focus group use with low-income minority populations is one described by Robin Jarrett (1993). Jarrett's study describes the implementation often focus groups in a study of 82 low-income, African-American women in the United States and concentrates specifically on the interactional dynamics that characterize both the recruitment process and the group discussion itself In discussing the recruitment process Jarrett asserts that while generally participants are invited to participate in focus groups through a process involving random sampling, she instead drew on personalistic strategies often associated with more intensive qualitative methods because of the nature of the study and the characteristics of the population (Jarrett, 1993). The focus group discussion described by Jarrett was informal and conversational but maintained a data gathering purpose through the use of a topical outline which was flexible enough to discuss issues in "the language of the women" (Jarrett, 1993, p. 188). Analysis of 22 the discussion provided insight into the dynamics of the group discussion, the establishment of rapport, the presence of' jperforming," and the intensive examination of participants' viewpoints. Jarrett says that for some women the focus group discussion had the 'feel of a rap session." In addition, with the presence of others, an "audience effect" frequently occurred. That is, members performed for each other and this performing encouraged women in the groups to discuss issues with great licence (Jarrett, 1993) . Jarrett's description of this performance aspect of the focus group with African-American women is one example which illustrates the need for further study of the implications of language and culture in focus group discussions. While discussion of the language used in focus groups is not available in the literature, use of language is commonly discussed in ethnographic studies of specific populations. One such study by Shirley Brice Heath (1983) focuses specifically on the cultural aspects of language use and the role of language in school "success." Heath's work includes extensive ethnographic descriptions of language use in informal gatherings of people in the African-American working class community of Trackton and the neighbouring white working class community of Roadville. Heath documents the presence of "performance" as a salient aspect of interaction among the African Americans and the absence of such performance in the white community. As well, Heath documents the various uses of storytelling in both communities. One example cited is the use of storytelling in the white community for purposes of teaching morality to children and the absence of morality tales in the African-American community. These examples of use of language, while not situated in a formal focus group environment, 23 do illustrate the importance of analysis of language and culture in focus group discussions. Gladstone (1972) states: Language and culture are inexorably intertwined. Language is at once an outcome or a result of the culture as a whole and also a vehicle by which the other facets of the culture are shaped and communicated. (P. 192) Crick (1976) argues that language is not only a means of communication but also a conceptual organization. Crick analyzes the field of modern anthropology and discusses the impact of linguistics and anthropology on each other in developing the idea that language determines perception and that language is a symbolic organization which is understood in terms of "meaning" and not in solely structural terms. Others state that there is a relationship between language and social behaviour (Robinson, 1972) and that language reflects the culture of the speaker and reveals different assumptions, beliefs and values concerning human and physical reality (Brislin & Yoshida, 1994; Munhall & Oiler, 1986) . Cultural concepts such as orientation toward space and time, social hierarchy, family and personal relationships, individuahsm versus collectivism, work and play are all concepts expressed through language. Use of Language in Research Processes As is evident in the literature, the involvement of language use in research and evaluation processes is not frequently addressed by researchers. Instead, most research simply omits 24 language as a variable (Kirkman-Liff & Mondragon, 1991) . However, there is some research which addresses the inclusion of non English speakers in research samples and works towards the development of processes to address the particular methodological issues that are inherent when working with non English speaking populations. Robinson (1972) describes a number of systems used to describe the functions of language. While Robinson describes these systems as inadequate to describe the functions of language completely, he classifies the functions of language as including but not limited to showing agreement, showing satisfaction, giving suggestions, giving opinions, asking for opinions and suggestions, disagreeing, asking for help and showing antagonism. Mohan (1986) also outlines a framework to address issues of language and meaning and to describe the various structures of language required for the variety of language functions. Mohan describes the thinking processes involved in comparing and grouping or classifying as well as the cause and effect principle which includes inferring, predicting, formulating hypotheses and generalizing about cause and effect relationships, and then identifies the language structures that are required for each of these language functions. Mohan illustrates a process of evaluation and suggests that the more elaborate language structures required for evaluating are generally learned later in the English language acquisition process. Research on language acquisition supports this notion of sequences or stages in the development of certain structures and the differences between second language learners from diverse cultures is less striking than the similarities (Lightbown & Spada, 1993) . For example, learners pass through similar stages in learning the negative elements, "ho" and "don't", and in learning question formation. Brislin & Yoshida (1994), in a discussion of language teaching, observes that language learners who are not confident of either their language ability or their cultural knowledge tend to be quiet and spend their time simply observing. Brislin & Yoshida conclude that this strategy tends to make the learner invisible to the host. In addition, "even when a more advanced stage comes to dominate in a learner's speech, conditions of stress or complexity in a communicative interaction can cause the learner to 'slip' back to an earlier stage" (Lightbown & Spada, 1993, p. 66) . Program participants who choose to participate in the Nobody's Perfect Program conducted in languages other than English do so for a variety of reasons. While some participants speak little English, others feel that the level of English they understand or speak is not adequate to discuss complex and personal issues. Others prefer to discuss parenting in their first language with those from their own culture regardless of their particular level of English competency. Brislin's (1981) discussion of studies of the effects of language fluency on newcomers indicates that people without the ability to speak the host country language limit the interaction they have with members of the host country. These limitations of interaction include limited involvement of non English speaking newcomers with program planning and evaluation. For researchers interested in the opinions of program participants, it becomes imperative to address this limited communication pattern. Each of the aspects of evaluation is important 26 including planning, data collection, data analysis, and reporting, and each of these processes becomes interwoven with issues of language, how people use words to mediate and construct reality, translation and cultural sensitivity. It is important that participants express their ideas about the program through the language in which they are most able to participate in evaluative discourse. Mohan and Schwab (1997) describe language as a "resource for meaning and cultural participation" and recommend greater attention to research in language learning and the role of discourse in institutional change. In an address to the International Conference on MulticulturaUsm and Minority Groups, they provide an example of female English as a second language students who are placed in high school physical education classes even though many female immigrants have negative attitudes about P.E. The authors suggest that the limited ability of the students to express feelings and reasons for judgments in English has restricted their effectiveness in influencing school policy. Students may be perceived by school officials as 'Venting feelings" rather than ''making a case for change." To enable students to participate in the institutional change process, the authors recommend providing opportunities for ESL students that will elicit their attitudes and judgments about P.E. as well as help them to identify the advantages and disadvantages of proposed solutions. Evaluative discourse may also be difficult for first language English speakers. Martin (1992) provides an example of a student's written response to a literary work. The student uses words such as "afraid" and "enjoyed" to express personal feelings evoked by the work but 27 fails to use the language of literary judgments. Therefore, the examiner judged these personal reactions as an inadequate expression of understanding of the literary work. The examples provided by Mohan and Schwab (1997) and Martin (1994) suggest the complexity of evaluative discourse including the expression of attitudes, personal feelings and judgments. To address this complexity and to involve non-English speaking program participants in evaluative discourse, it is important that participants be given opportunities to express themselves in the language in which they can communicate most effectively. The choice of using focus groups in participants' first language as the methodology for the evaluation of the Nobody's Perfect Program was based on the intention to include non-English speaking program participants as stakeholders in the evaluation process and to gather information from them in their preferred language. There was no need to determine the level of English competency of participants on any kind of language test or other measure. While such measures of English language competence are relevant to some situations, they are unlikely to predict language readiness or the comfort level of individuals to participate in English in a structured conversation such as a focus group. 28 CHAPTER THREE METHODOLOGY The Case Study Approach The study of the utility of focus groups as an evaluation tool for non-English speaking stakeholders is important because of what may be learned about the suitabiUty of focus groups for evaluation purposes with this particular population. A descriptive case study approach was selected as the research methodology because the specificity of focus of the case study makes it an especially good design for practical problems-for questions, situations or occurrences arising from everyday practice (Merriam, 1988) . The case study is suitable for addressing a specific phenomenon where understanding is sought in order to improve evaluation practice (Merriam, 1988; Stake, 1995; Yin, 1994) . Merriam (1988) states that while the use of a case study approach is not new to the field of education there is little material which addresses the actual methodology of case studies. Recent works by Yin (1994) and Stake (1995) have discussed the use of the case study approach for evaluations such as the Nobody's Perfect Program evaluation. Yin (1994) discusses the use of the descriptive case study in evaluation research to describe or illustrate specific topics within an evaluation context. Within this context, the case study approach for this thesis was used to explore the utility of focus groups with program participants who did not speak English. This approach enabled the researcher to gain an in-depth understanding of 29 the complexity of information available through the focus groups conducted in participants' first language. Stake (1995) points out that the knowledge gained through case studies can be concrete and contextual and can contribute to an understanding of specific aspects of programs being evaluated. In the study of the Nobody's Perfect Program, the case study approach formed the basis of understanding the utility of focus groups in obtaining evaluative information from non-English speaking program participants. Merriam (1988) suggests that descriptive case studies are useful in presenting information about areas of education where little research has been conducted, that innovative programs and practices are often the focus of descriptive case studies in education and that such studies can form a base for future comparison and theory building. Yin (1994) identifies five different applications for case studies in evaluation research. These include explaining causal links in real-life situations that are too complex for survey or experimental strategies, describing an intervention and the real-life context in which it occurred, illustrating certain topics within an evaluation from a journalistic perspective, exploring situations where the intervention being evaluated has no clear, single set of outcomes, and a meta-evaluation-a study across evaluation studies. The case study undertaken for this thesis would most clearly fall in the second category described by Yin, which is a description of an intervention. The intervention being described is the six focus groups that were conducted with recent non-English speaking participants of the Health Canada parenting program, Nobody's Perfect. This case study describes six focus groups, two 30 focus groups to involve program participants and to provide new and useful information for program planners and policy makers. The design of this case study involved implementing a logical sequence that would connect the data to the study's research question and to its conclusions (Yin, 1994) . The process involved a number of components which included developing the focus group questions, identifying and working with bi-lingual and bi-cultural focus group facilitators, inviting participants, conducting the focus groups, translating the focus group transcripts, analyzing the data and developing conclusions. Each of these aspects of the process is described in this and later chapters of the thesis. Developing the Focus Group Questions Focus group questions were initially developed by the researcher in English and then refined through a consultative process with the Evaluation Steering Committee from the agency sponsoring the program evaluation of the Nobody's Perfect Program As the implementation of the parenting program with non English speaking participants had not been included in previous program evaluations the committee hoped to learn about the experiences of non English speaking participants. Questions were intended to enable program planners and funders to obtain comprehensive information that was previously unknown. 31 The intent was to develop a set of questions that would encourage the discussion required to elicit information about participants' experiences and reactions to the Nobody's Perfect Program Seven areas of interest were explored: 1. Why did you choose to participate in Nobody's Perfect? 2. What did you like about the program? 3. What did you learn? 4. Did the leader of the Nobody's Perfect Program seem more like a teacher or a facilitator? 5. Did the Nobody's Perfect Program seem appropriate to your culture? 6. How were the Nobody's Perfect books useful to you and what suggestions do you have for improvement? 7. What ideas do you have for improving the Nobody's Perfect Program? Focus group interviews cannot be pre-tested in the manner used by mail out or telephone surveys where some people from the intended audience are selected for a pilot test. For focus groups the process of detennining appropriate questions must consider a range of factors such as the information required, the characteristics of the participants and the interaction between participants and the facilitator (Krueger, 1994, p. 68) . Accordingly, the focus group questions were reviewed by program staff familiar with both the Nobody's Perfect Program and the evaluation goals. It was acknowledged that the questions could not be fixed, as they would be in a conventional survey for example, but, rather, would evolve during the group process. It was the responsibility of the focus group facilitator to encourage group process in such a way 32 that participants would feel comfortable in sharing their experiences and opinions about the program. Facilitators would begin with the fixed set of questions and would also be ready to alter the questions as required to elicit the desired information from the group. Focus Group Facilitators The importance of the focus group facilitator in the collection of data is paramount (Basch, 1987; Krueger, 1994; Morgan, 1993) . Focus groups provide an environment where open discussion is encouraged and nurtured, but it is the facilitator who ensures a focused discussion (Krueger, 1994) . The data are the result of the interaction between and amongst the facilitator and group participants (Stewart & Shamdasani, 1990). Most of the literature concerning focus groups and the collection of qualitative data suggests an advantage in having the researcher moderate the group directly (Merriam, 1988) . After all, the researcher is most familiar with the research topic and with the research questions. However, in this case study, where focus groups were to be conducted in several different languages, it was necessary that the English speaking researcher work with bilingual, bicultural facilitators from several language groups. The facilitators became key players in the focus group process, strengthening the process by addressing a variety of cultural perspectives (Berrien, 1967; Bornstein, TaL & Tamis-LeMonda, 1991; Slaughter, 1991). For the purpose of the implementation of the focus group discussion these individuals took on a "co-researcher function" or acted as what Pomerleau, Malcuit and Sabatier (1991) call "co-linguistic interviewers." 33 The literature addresses the contributions that can be made by researchers working together in various roles. Anderson (1990) suggested that researchers from diverse cultures should be involved to assist in explaining the research to participants, in obtaining informed consent and in interpreting research results. Bornstein et al (1991) , identified benefits from using "cultural natives" to make observations. Krueger (1994) says that such persons may have skills, connections, energy, and ideas that can enhance the potential of the study. He asserts that when the facilitator is trusted and the study is considered acceptable to local participants, the focus group can be conducted in a variety of successful ways. For the focus groups implemented in this evaluation of the Nobody's Perfect Program, bilingual, bicultural focus group facilitators provided the mechanism by which the voices of non English speaking focus group participants could be heard by the researcher. Successful selection of focus group facilitators was largely the result of the enthusiasm with which involvement of non English speaking program participants in the evaluation was welcomed by agency staff and program participants. Participants were pleased to have the opportunity to offer their views of the Nobody's Perfect Program which had been popular within their cultural communities since 1989. Both agency staff and program participants welcomed the chance to raise issues specific to their particular experiences with Nobody's Perfect in languages other than English. 34 Agency staff identified several individuals with the required language skills who also were respected within their cultural communities, understood the Nobody's Perfect Program, and possessed the necessary facilitation skills to act as focus group facilitators. Typically, candidates included individuals who were settlement workers or instructors at one of the agencies offering the Nobody's Perfect Program. It was necessary that focus group facilitators be credible individuals who were trusted by focus group participants. The importance of trust between the facilitator and group participants cannot be underestimated. Issues of crechbility and trust were carefully considered and consistently indicated as key components in the choice of individuals for the facilitator role. This notion of trust is discussed in the literature in the context of how researchers establish trust and the importance of personal relationships in gathering data in social research (Lincoln & Guba, 1985) . It also became evident that trust and relationship building were viewed in diverse ways in various cultural communities and while this diversity is an exciting area for future research, it was not the focus of this work. The individual experiences and circumstances of individuals chosen as facilitators varied. For example, the Spanish speaking facilitator was an individual who was working with groups in a number of settlement organizations in the city. She was familiar with the goals and content of the Nobody's Perfect Program through discussions with staff at those settlement agencies. She was described by program staff as someone who was known to, and trusted by, "the Spanish speaking community" through her various political and social connections. The Chinese speaking focus group facilitator had not worked in parenting programs but was experienced in the child care field and thus considered familiar with similar issues. She came 35 highly recommended from a variety of sources for her knowledge, experience and facilitation skills. The Vietnamese and Punjabi speaking facilitators were chosen from among trained Nobody's Perfect facilitators as no other candidates could be identified. Facilitators who were familiar with parenting programs but not directly involved with Nobody's Perfect would have been preferred because there was a concern that focus group participants might not be as frank in their discussion if the focus group facilitator was the same mdfvidual who had facilitated their own Nobody's Perfect group. To offset this, Vietnamese and Punjabi speaking focus group participants were selected from a number of different Nobody's Perfect groups which had been conducted in those languages. However, there is no way to know what effect familiarity with the facilitator might have had on the data gathered from those particular groups. The researcher met with each facilitator to provide a general orientation to the philosophy and goals of the Nobody's Perfect Program and to review the focus group questions. There was no attempt to train these individuals in group facilitation or group process as each of the individuals had been chosen specifically for their identified skills in those areas. Rather, the researcher's intention was to ensure that each facilitator had sufficient understanding of the Nobody's Perfect Program and of the evaluation goals to guide the discussion as required. Additionally, the researcher and facilitators identified cross cultural issues related to the philosophy, content and materials of the Nobody's Perfect Program and the translation of the focus group questions. 36 Cross Cultural Facilitation of Focus Groups The focus group question guide prepared by the researcher was reviewed by each of the group facilitators. Specific cultural issues related to the Nobody's Perfect Program were identified by each of the facilitators. For example, discussions with the Chinese speaking facilitator about the title of the program, "Nobody's Perfect," raised some significant cultural issues. The meaning of this term had long been a great source of discussion and learning for those interested in the cultural relevance of this parenting program to the Chinese community. Nobody's Perfect was chosen by Health Canada as the program title based on the principle that no one is a perfect parent, that parents need only try to be the best parent they can, and that no one should expect perfect children. This principle is contrary to a strong tenant of Chinese culture which does not have a verbal expression to indicate that it is desirable for an individual not to be perfect. Rather, the culture encourages people to strive for perfection and has a clearly articulated set of criteria which describe ideal parenting and ideal children. This discussion identified an area to be explored in the focus group discussions with Chinese speaking parents. The discussion between the researcher and the Spanish speaking facilitator focused on the relevance of the philosophy of the Nobody's Perfect Program to Latin American culture. In the Spanish speaking Nobody's Perfect groups, one of the issues which had often been raised was the role of the mother and the father in parenting and to what extent these roles changed with immigration to Canada. As a result, this community had frequently directed some Nobody's Perfect groups specifically to couples. Therefore, this research included two focus groups for Spanish speakers, one for parents from "couples only" Nobody's Perfect groups and one for parents from other "mixed" Nobody's Perfect groups. The planning process between the researcher and the Vietnamese facilitator also highHghted a cultural issue. Although only one individual had been selected for the facilitation role, when the researcher arrived at the agreed upon meeting place, two women were present. One of the women was the facilitator previously identified in the selection process and the other was a trained Nobody's Perfect Program facilitator and colleague of the first woman. The two women explained that they intended to co-facilitate the focus group as they had done in the Nobody's Perfect groups. While co-facilitation of focus groups had not been in the research plan, there seemed no reason to oppose the idea. Further consultation with knowledgeable professionals provided the cultural context for this occurrence. Because we had approached the facilitator who was somewhat less experienced than the other, she was obliged to invite the other facilitator to work with her. This action would maintain the "balance" between the two facilitators in the eyes of their community and between the facilitators and the researcher. After the discussions between the researcher and the focus group facilitators, the focus group questions were translated by each facilitator and a copy of the translation given to the researcher. It was the role of the facilitator to facilitate the group process in such a way that participants would be invited and encouraged to share their responses to the topics outlined in 38 the focus group questions. The facilitator would ask the predetermined questions in ways that would elicit the information required for the research. One issue frequently discussed in the literature pertaining to cross cultural research is the comparability of research instruments (Berrien, 1967) and the equivalence of measurement primarily related to the equivalence of various types of measures used in psychological tests (Adler, 1977; Bornstein et aL 1991; Hui & Triandis, 1983) . While these issues are clearly important for quantitative research methods, they are not of primary importance in the analysis of the qualitative data obtained from the focus groups in this study. As focus groups are discussions among groups of people and as these discussions vary from group to group as well as from culture to culture, the comparability of research instruments and the equivalence of measurement are not central methodological issues. Rather, the translation process of the focus group questions and the translation of the data for the English speaking researcher are of primary importance (Brislin, 1976) . Focus Group Composition Former Nobody's Perfect Program participants were invited to participate in one of six focus groups according to their preferred language. Two focus groups involved Cantonese speaking parents who had participated in Nobody's Perfect in Cantonese and two other focus groups involved Spanish speaking parents from Nobody's Perfect groups conducted in Spanish. A fifth focus group involved Punjabi speaking parents and the sixth focus group was conducted in Vietnamese. It was decided to hold two groups in Chinese and Spanish because of the higher enrollment in Nobody's Perfect groups offered in these two languages as compared with Punjabi or Vietnamese groups. The composition of focus groups is frequently discussed in the literature (Stewart & Shamdasani, 1990; Morgan, 1993; Krueger, 1994; Morgan, 1997; Vaughn, Schumm, & Sinagub, 1996). This discussion focuses primarily on the "strangers versus acquaintances" composition and heterogeneous versus homogeneous composition. Generally, the accepted guidelines favour groups composed of "homogeneous strangers" and several authors offer suggestions as to how to achieve this composition. For example, in a recent work, Morgan (1997) discusses the recruitment of strangers and describes a process of segmentation in which potential participants are matched according to specific categories such as gender, age, or race. He suggests that such homogeneity is sought to promote free flowing conversation among people who feel comfortable with one another. At the same time, Morgan (1997) posits that the notion that focus groups must consist of strangers is a myth. Instead, both Morgan (1997) and Morgan and Krueger (1993) argue for a kind of realism where researchers must account for "real life" situations in which acquaintanceship in naturally occurring groups is unavoidable and often beneficial to the investigation. Also, both Morgan and Krueger acknowledge that there are many examples of groups which seem to "work" perfectly well with mixed genders, or races, or other 40 characteristics. In an article describing focus groups which discussed doctor-caregiver relationships, Morgan (1992) argues for informed design choice rather than adherence to a specific set of rules for focus groups. Both authors assert that the important criterion to consider is whether people can comfortably discuss a topic in a way that is of interest to the researcher. For the focus groups discussed in this thesis, the primary considerations for participant selection were completion of the Nobody's Perfect Program delivered in the participants' first language and a willingness to be an active part of the focus group discussion. In this context researchers saw no need to gather further demographic information from participants and in fact, agency staff suggested that asking for such information would severely inhibit program participants from agreeing to participate in the study. Instead, researchers made an assumption of homogeneity among focus group participants based on the criteria outlined for participation in the Nobody's Perfect Program Participants in the Nobody's Perfect Program are parents of children from newborn to six years of age who need support in parenting their children because of conditions related to limited income and education, because they may be young or single parents and because they may be socially, geographically or culturally isolated. A significant commitment and considerable effort were required from the agency staff to ensure an effective recruitment process. This involvement in the recruitment of focus group participants and the organization of groups ensured that most participants who had been recruited did actually attend the focus group (Morgan, 1997) . Typically, informants 41 participate in qualitative field studies because of their relationship with the researcher, (Jarrett, 1993) however, individuals participated in this study because of their relationship with agency staff or the Nobody's Perfect group facilitator who invited them to attend the focus group. Each potential participant was contacted by agency staff or group facilitator on the telephone or in person and invited to be a part of the focus group. All discussions regarding the purpose of the group, expectations of the participants and logistic considerations were conducted in the individual's first language. These initial contacts by agency staff and facilitators produced considerable interest in participation in the focus groups resulting in 23 people attending the two Chinese language groups (12 in one group and 11 in the other group), 8 attending the Punjabi language group, 22 attending the two Spanish language groups (11 in each group) and 13 attending the Vietnamese language group. In summary, 66 people attended the 6 two-hour focus groups. Permission To Participate It is required that participants in any research study sign a document agreeing to participate and indicating that they understand the nature of the study as well as their rights and responsibilities (Anderson, 1990). Focus group participants were invited to participate in an evaluation of the Nobody's Perfect Program (see Appendix) . In consideration of the need for participants to provide informed written consent, focus group facilitators provided an oral word-for-word translation of the English document prepared by the researcher and then answered questions from participants as needed. After this process, all participants signed the 42 document, retained a copy and gave one copy to the researcher. Ideally, this document would have been translated into each of the participants' first language. However, budget implications precluded such translation. Data produced in the context of the program evaluation of the Nobody's Perfect Program included transcripts of the six focus groups conducted with non English speaking program participants (Ritch & McLaren, 1994) . The researcher conducted a secondary analysis of these transcripts to study the utility of these focus groups to produce new and useful information for program planners and policy makers. As Thorne (1994,) suggests, "analytic expansion, in which the researcher makes further use of his or her own original database to answer questions at the next level of analysis or to ask new questions" (p. 266) is an appropriate rationale for the use of secondary analysis of qualitative data. Support For Parent Participation The researcher worked with parents and agency staff to determine the specific supports needed to enable them to participate in the focus groups. These supports included convenient locations and time of day and concurrent on site child care. The six focus groups were held in locations determined to be the most comfortable and accessible for participants and at times most convenient for parents of young children. Past experience with the Nobody's Perfect Program indicated that many participants were reluctant to travel outside of their own 43 neighbourhoods. Often people who relied on public transportation did not have the required bus fare and so for all six focus groups, bus fare was provided as needed. Caregivers for participants' children were hired from the available list of childminders who had provided care to children at Nobody's Perfect groups. This meant that the parents and children were familiar with the caregivers and parents felt assured that their children were well cared for during the focus group session. In all cases, at least one of the cMchninders spoke the language of the parents. This was crucial to facilitate the smooth transition of parents leaving their children to participate in the focus groups and to ensure the comfort of the children in the child care setting. Participants were offered an honoraria of $25.00 as one way of expressing the value placed by the researcher and the sponsoring agencies on participants' time. These honoraria were distributed at the completion of each focus group session. Conducting the Focus Groups The focus group facilitators were responsible for promoting the group discussion necessary to respond to the topics outlined in the focus group questions. The role of the researcher during the focus group was primarily to operate the tape recorder and to provide other assistance to the facilitator. At the beginning of each of the six focus groups, the researcher was introduced to the group as the individual who was responsible for the evaluation of the Nobody's Perfect 44 Program. During the focus group process, the researcher observed the group but no attempt was made to document the process itself. At the conclusion of each focus group, the researcher and the facilitator discussed the group process in relation to how well the focus group questions had elicited evaluative comments from participants. Recording, Transcription and Translation Of Focus Group Data Each focus group was audio taped using a tape recorder and a PZM multi-directional microphone specifically designed for use in focus groups. This equipment produced a clearly audible tape of the focus group discussion. Written transcripts of the audio tape were prepared by the focus group facilitators, all of whom had previous experience preparing transcripts. These transcripts were translated into English by experienced bicultural translators with knowledge of the area of parenting and family issues (Casagrande, 1954; Slaughter, 1991). The process for the translation of the focus group transcripts was a crucially important aspect of this evaluation because the analysis of participants' responses was based on the English translation of the transcripts prepared from the audio taped focus group sessions. While Krueger (1994) identifies that transcript-based analysis of focus groups is extraordinarily slow and cumbersome, the element of translation of data from one language to another necessitated a reliance on accurate transcripts. Also, as the discussion in focus groups depends heavily on the facilitator and since there is some uncertainty in the field as to 45 standards which should apply to interpretation of qualitative data, it is important to record the group and to prepare transcripts that can be analyzed and reanalyzed (Basch, 1987) . One of the primary issues related to conducting focus groups in languages other than English is the requirement for accurate translations of focus group discussions. Brislin (1976) , whose work has been in the area of the use of translation as a research tool in cross-cultural studies, defines translation as: The general term referring to the transfer of thoughts and ideas from one language to another language whether the languages are in written or oral form; whether the languages have established orthographies or do not have such standardization; or whether one or both languages are based on signs, as with sign languages of the deaf. (P-1) One technique often described as effective towards ensuring accurate translation is a technique called "back translation". This technique involves two bilingual individuals-one who translates from the source to the target language and the other who translates from the target language back to the source language (Brislin, 1970). The authors suggest the following process for back translation: the investigator prepares the interview schedule and two translators work on it, each translating half into the target language. Then each takes the other's work and translates it back to English. The researcher then has two versions of the English text and through them a triangulation on to the target language. This process highUghts any discrepancies in the translation. Regarding back translation, Werner and 46 Campbell (1970) suggest that this technique is very useful when the researcher is interviewing through an interpreter and when the researcher knows little of the target language. The researcher is in a "helpless" situation and the use of back translation offers some degree of discipline. Others suggest that while back translation usually uncovers considerable difficulties in the translated material, researchers should view this as the strength of the technique-it confronts the researcher with the difficulties of researching in contexts where the researcher is not familiar with the target language (Sinaiko & Brislin, 1973; Werner & Campbell, 1970) . Back translation is also suggested as a method by which the researcher may assess the competency of the translators (Werner & Campbell, 1970) . Other techniques described for translation include the use of bilingual individuals, pretesting to complement back translation and the use of numerous translators to arrive at the best possible translation (Ervin & Bower, 1953; Prince & Mombour, 1967) . In these processes, if there is consensus or a high level of agreement on a single version, one might assume that it is an accurate translation. "Consensus is, after all, the ultimate arbiter of linguistic usage" (Casagrande, 1954, p. 339). The researcher worked with community agencies to identify translators in each of the focus group languages with translation experience in the content area of family issues. While the researcher recognized the strengths of back translation, and while this technique would have made a contribution to the accuracy of the translations, back translation was not conducted primarily due to its cost implications. Translation by more than one translator was also considered but not used because of budgetary restrictions. 47 Both the inherent complexities of language which describes diverse parenting approaches and practices and the number of languages being translated into English were significant factors in the selection of a translation process for the focus groups described in this study. Community groups involved in the translation of print materials related to issues of family, child development and child care have identified that translators' familiarity with particular concepts and terms used in these content areas is as important as accuracy of the language. This knowledge and understanding is known to contribute to greater accuracy and more meaningful translations. This was the primary consideration in the selection of a translation process for focus group transcripts. Where transcripts contained sentences which were not grammatical in English, these were brought into standard English for clarity and easy reading. There seemed no advantage to using flawed English which resulted from a limitation of the translation and not a limitation of the focus group participants. 48 CHAPTER FOUR ANALYSIS OF FOCUS GROUPS Understanding Participants' Response to the Nobody's Perfect Program The preferred approach for the analysis of data obtained through a case study is to interpret the findings in light of the theory which led to the case study (Yin, 1994; Merriam, 1988) . One is encouraged to consider the original objectives and research questions upon which the case study was designed (Yin, 1994) . For the purpose of the research described in this thesis it was established that the literature described focus groups as an effective tool to involve participants as stakeholders and to obtain considerable information from focus group participants regarding their experience. It was also established that while the literature encouraged researchers to use focus groups with "special" populations, this method had not been used with non English speakers. The analysis of the focus groups with non English speaking program participants implemented for this study examined the extent to which the discussion produced information for program planners and policy makers which was not previously known and which was useful for program evaluation. Focus group transcripts translated into English were analyzed using a descriptive approach for organizing results based on the set of focus group questions used in each of the focus groups (Yin, 1994) . The primary purpose of this approach was to organize the responses of the participants to the structured set of questions. Then, an analysis grid was developed to 49 systematically code the focus group participants' responses to each of the focus group questions (KnodeL 1993). Two categories were developed to enable the researcher to determine the substantiveness of participants' responses to the questions. These categories served as an heuristic device to analyze and describe the complexity of responses from focus group participants and to 'illuminate the reader's understanding of the phenomenon under study" (Merriam, 1988, p. 13). Transcripts were reviewed and each statement coded as either category one, "simple responses" or category two, "elaborate responses." Simple responses were those where the responses provided basic information with little or no detail. Elaborate responses were those where participants provided enhanced descriptions, specific details or complex explanations. For the purposes of this thesis, no attempt was made to systematically analyze either similarities or differences between the various focus groups or language groups. Morgan (1997) suggests that when a study design involves several groups which are highly segmented in their composition, the analysis process often invites the researcher to make comparisons between the various groups. Knodel (1993) suggests, however, that in spite of this invitation, researchers are advised to concentrate on drawing conclusions based on similarities rather than those based on differences. He suggests that when similar views are expressed by various groups, it is likely that these represent shared views and it may be difficult to distinguish differences among groups that may not be attributable to factors such as the way the group was facilitated, particular group dynamics or personalities of the individual participants. In spite of this caution, analysis of the focus groups done for this thesis indicates that there are 50 certain cultural factors which may influence the implementation and results of focus groups and fiuther research in this area is strongly advised. As could be expected in any focus group, there were both simple and elaborate responses to each question. However, the vast majority of responses in these focus groups provided important and relevant information that was previously unknown to program planners and policy makers. These responses provided much insight into the attitudes, perceptions and opinions of participants (Krueger, 1994) and permitted the study of selected issues in depth and detail (Patton, 1990). This research confirmed anecdotal reports from those working in programs with limited English speaking participants that while participants will often sit silently during discussions conducted in English, these same individuals will enthusiastically offer extensive feedback when they have opportunities to discuss their experiences using their first language. The findings regarding the Nobody's Perfect Program that resulted from analysis of the focus groups discussions in languages other than English substantiate the utility of focus groups for purposes of program evaluation. None of the individuals participating in these focus groups would have been able to provide such in-depfh iriforrnation in English. In addition, the focus group process suggests that the secure setting of group discussion mvorving participants who were familiar with one another contributed to the uncovering of important information for program planners and policy makers. The remainder of this chapter describes the extent to 51 which participants provided substantive responses to focus group questions and suggests ways that the discussion process itself contributes to particpants' responses. Why Did You Choose To Participate In Nobody's Perfect? When focus group participants were asked to discuss why they had chosen to participate in the Nobody's Perfect Program, they offered a variety of explanations. Of the 49 separate responses to this question 14 were coded as category one while 35 responses were considered "elaborate" and coded as category two. Elaborate responses from English as a Second Language parents are rare when feedback is invited in English, however, when given opportunities to express opinions in their first language, these parents provided valuable information for program planners and policy makers as to their reasons for participating in the Nobody's Perfect Program In category one responses, parents briefly described their reasons for attending the program Typically these parents reported that they attended because they wanted to learn more about parenting and the program was available in their first language. As basic as these responses are, it is likely that they provided more information than these parents could have provided in English. This confirmed for planners and policy makers the importance of providing programs in parents' first languages. The following are typical of category one responses to this question: 52 Since my kids were at the right age I attended this group for my personal benefit and for the benefit of my children. I wanted to learn how to treat my children in a different way. I would like to know how to educate my children and how to prevent them from danger. Staff at the Neighbourhood House told me they were doing a parenting group in Punjabi so I came. Typical category two responses contrast sharply with the previous examples in terms of the complexity of the ideas expressed and the richness of information given. These responses describe parents' awareness of the need to understand Canadian culture to better understand the cultural influences faced by their children. Program planners and policy makers were provided with information which indicated the concern parents had for the successful integration of their family into Canadian culture. Previously, some program planners had suggested that parents did not want to be influenced by Western parenting practices and that distinct parenting programs for diverse cultural groups needed to be developed. As each parent in the focus groups discussed their concerns, others followed with similarly elaborate descriptions of their own concerns. The following provides examples of the richness of the discussion related to this topic: 53 We are in a new culture with different patterns and values. We wanted to understand how our children were going to be and to try to understand them better in their new world. We wanted to learn how to deal with the new ways because we are aware that the outside is going to have a major influence in our children's life. Now I am living in Canada, and I have two children who will grow up in the Canadian culture; therefore, I would like to learn this program in order to educate my children. The reasons which encourage me to attend this program are: I want to learn more about how to educate children in the Western culture; I also want to compare the differences between the Western and Oriental education for young children. We are in a new culture with different patterns and values. We want to understand how our children are going to be in their new world. We wanted to learn how to deal with the new ways because we are aware that the outside is going to have a major influence in our children's life. No matter if you are a new or old immigrant to Canada, our upbringing and backgrounds are different from the Western culture. We hope to be able to know how 54 to raise our children in the Western culture and how to communicate better with them. The old Chinese method of child rearing may not be completely suitable for today's needs. Some ideas are not applicable anymore. This motivated me to come to Nobody's Perfect. We are always interested in knowing about our children's development. When we arrived we did not know how to guide our children in this new environment. We didn't know what Nobody's Perfect was all about but while doing it we liked it a lot. We would have like to have gotten deeper into some topics but there was not enough time. These parents' complex statements helped program planners and policy makers to understand more fully the motivations and personal goals of program participants. Although, as in the first group, the primary motivation expressed was a desire to learn about parenting, these respondents expanded upon this basic idea to offer reflections of their particular challenges related to parenting in western culture. These parents identified an intense desire to guide their children appropriately in a culture which they themselves did not fully understand. This was new information for program planners who previously had been unaware of the degree to which immigrant parents felt it important to learn "new ways" in order to feel successful in parenting in a new environment. This expanded understanding of the settlement process led to the decision not to proceed to develop other parenting programs for newcomer parents. 55 Rather, program planners understood the need to promote discussion related to parenting in a "new" culture within the current structure of the Nobody's Perfect Program The expression of the complexity of the settlement process and the extensive information provided through the focus groups was possible because the discussion was in parents' first language and involved a discussion process in which parents felt comfortable to discuss their struggles in the new culture. What Did You Like About The Nobody's Perfect Program? There were a total of 36 responses from parents to this question. Nearly 90% of these were category two elaborate responses. In all of the focus groups, this question seemed to prompt parents to discuss benefits they had derived from the program and to some extent the structural elements of the program itself including the provision of on-site child care, program books for each participant and identification by parents of favourite topics of discussion. For the most part, however, parents' comments addressed the benefits derived from the program as well as specific information parents had learned in the program that improved their parenting. Discussion vvithin the context of each of the focus groups produced information about a range of topics. In one group most of the participants expressed a personal reaction to the program facilitator. When one participant expressed a dislike for the facilitation style of the mdfviduaL each of the other participants expressed a personal reaction to the facilitator, some positive and some negative. As well, in response to the question regarding what they liked about the program, each of the participants commented on aspects of the program such as the sharing of information among participants and the reduced sense of isolation as a family. In another group, when one individual mentioned liking the visit from the Public Health Nurse and the discussion about children's health, there followed a discussion in the focus group about the positive and negative aspects of inviting "professionals" to meet with parents as part of the Nobody's Perfect Program This discussion became related to the role of the facilitator in the program which was actually a later planned question in the focus group process. Because parents were comfortable in their spoken language, the discussion could develop naturally with participants responding both to the questions asked by the facilitator and to comments made by focus group participants. In another group the first respondent mentioned the helpful sharing of information among first-time mothers. Subsequently, each of the other participants addressed the aspect of sharing information among group members. In this group a simple statement made by one participant regarding shared information was followed by six elaborate descriptions of the same aspect of the Nobody's Perfect Program The following comments from parents are examples of elaborate statements which indicated that the program was indeed successful in promoting the development of such support networks: 57 Staying home all the time taking care of our children is tiring and stressful and makes us grumpy. Coming to Nobody's Perfect we can share our ups and downs with each other. Having a group to lean on we become more happy and not so grumpy. We are quite lonely in Canada with few relatives andfriends around. Even if we have friends they are busy with jobs, house-work andfamily. We seldom get in touch with them. Coming to Nobody's Perfect, we can actively share, get support, and release our stresses. We are happier after joining the group. Before I joined the group, I was very quiet, very private and not used to sharing. Now I have learned to open up myself and to share my feelings with others. My husband said that I have become a new person. It is very important that the couple can do the program together, because there are a lot ofprograms offeredfor women but not for men. We met new people and created goodfriendships. I liked the Nobody's Perfect Program because I learned to educate my children and prevent them from danger at home or school. I liked everything in this program because I could share my concerns about my children. Also, I liked the group. We shared experiences when we got together and the group encouraged me to make decisions. I feel more relaxed. 58 These discussions in parents' first languages allowed program planners and policy makers to gain insights into aspects of the program which were most appreciated by program participants. More importantly, they also obtained detailed descriptions of the benefits derived from the group dynamics and from sharing experiences with other parents. Discussions were characterized by the 'building'' of the conversation which is a hallmark of focus group discussions (Krueger, 1994; Morgan, 1993; Morgan, 1997). Within the context of the focus group participants seemed to be honest about the stresses they had been experiencing vvithin their families and the contribution the Nobody's Perfect Program had made to their well-being. They noted the sense of isolation within their families and their communities and indicated that participation in the Nobody's Perfect Program had helped reduce some of that isolation. Program planners learned the extent to which the Nobody's Perfect Program was achieving its goal of buttding self-help and mutual support networks among non English speaking participants. What Did You Learn From The Nobody's Perfect Program? Responses to this question enabled program planners to understand more about what parents learned through participation in the Nobody's Perfect Program The 32 responses to this question provided information about practical, knowledge based learning related to child safety, child development and discipline. Because of the concrete nature of this question, it is not surprising that the proportion of simple and elaborate responses was fairly even with 59 category two responses accounting for slightly more that 50% of the total number of responses. The focus groups with non English speaking parents provided considerable information about how program information was used and what outcomes were realized from the program In both category one and category two responses parents described in some detail actual situations where they had applied the knowledge they gained from the Nobody's Perfect Program In most cases situations described involved incidents that compromised health or safety where parents were able to intervene resulting in a positive outcome for the child. However, elaborate responses extended this type of description to include expressions of parents' growing confidence in themselves as competent parents that stemmed from their ability to respond to their children's needs. These elaborate responses contained insightful descriptions of parents' understanding of their own children's needs as well as some discussion of how parents learned to meet those needs more effectively. As well, the complexity of the focus group discussions allowed the link between parental confidence and knowledge about safety and child development to be made clear. Information about the extent to which parents' sense of confidence had increased through greater knowledge of child development and safety demonstrated the importance of providing information to parents. Overall, in the secure environment of a group of parents with similar experiences, parents were able to express their difficulties and describe how the Nobody's Perfect Program had helped them be more competent and confident parents. The following 60 comments show examples of situations where newly acquired knowledge had a direct effect on both the child's well-being and the parents' self-esteem: My son complained of severe pain in his arm one time. I didn't know what the problem was. I wondered if it was a fracture. I remembered the first aid that I learned from Nobody's Perfect. I then used some cloth to tie around his arm and gave it support before taking him to hospital. It felt good to know what to do for him. I learned child safety along with taking care of them and loving them no matter what they do. And that we should be more understanding of them because they are just kids. When we talked about health and age development I started observing my child carefully. I realize that something was wrong with his ear. With the information I had I went to the doctor with my child. I was right and because it was early enough, my child is doing great. Through this program I can understand the emotions of my children, their ups and their downs. They are just like us adults, and they also need to release their stresses. 61 I learned that they have needs and that I should fulfill those needs to make them feel that there is support for them. I am a new immigrant. I don't have many friends or relatives around me. My daughter cried a lot, and I didn't know what to do to take care of her. For a while, I was so frustrated and bored with staying home taking care of a crying baby, that I wanted to quit. But the group gave me a lot of help and support and I can go on. Sharing experiences with other mothers I realized I was not a bad mother after all. Parents also noted that the Nobody's Perfect Program had influenced their approach to discipline and guiding their children's behaviour. In their first language parents were able to reflect on this complex aspect of parenting and eloquently express changes in their attitudes and their actions. Most parents identified an approach to discipline which included corporal punishment as an integral part of their cultural background. Some parents emphasized an abrupt change in their approach that resulted from the program while other parents' comments described a gradual transition from a traditional approach that utilized corporal punishment to a more "Western" approach characterized by verbal guidance techniques. Most of the comments seemed to indicate agreement with the notion that "Western" discipline was preferable. The following are examples of the statements made by many parents: 62 I learned to discipline my kids the way that we both can benefit and not just me yelling at them and they not knowing what they did wrong. You should teach them right and wrong things as you go along and not expect them to do the right things all the time. Sometimes we had unhappy times between wife and husband or parents and our children annoyed or disturbed me. I wanted to spank them but I realized what I have learned in the program and I stopped doing it. It is an excellent program. I learned to use reward and consequence systems on my children to encourage them to do good and to develop virtuous characters. I now stop and think over the whole matter when I am about to apply physical punishment on my son. I don't force him to do things my way anymore. Westerners are more concerned about manners and showing respect to their children. That is what we should learn from them. Did The Leader Of The Nobody's Perfect Program Seem More Like A Teacher Or A Facilitator? This focus group question related to the learner-centred model upon which the Nobody's Perfect Program is based. A basic principle of this model is that parents want to learn more 63 about parenting and can learn parenting and build a support network through participation in a group learning experience. The program literature describes this model as one which values the participation of all parents, where there is no expert or teacher and where the knowledge about parenting is shared through group discussion. The theory behind this approach stems from the emphasis on a health promotion approach which attempts to shift responsibility for health from the "expert" to the "consumer." The Nobody's Perfect Leaders' Guide lists six guidelines for the implementation of the learner centred approach: • Involve the parents in deciding what they want to learn in the program and how they want to learn it. • Create a friendly, safe, and non judgmental atmosphere. • Encourage discussion. • Create learning activities which enable parents to understand their situations and solve some of their own problems. • Be prepared to change your session plan to suit the needs and interests of parents. • Encourage self-help and mutual support. Because of the centrality of this model to the program design it was important for program planners to learn about the degree to which various groups adhered to this model and the perceived effectiveness of this approach by non English speaking parents from a variety of cultures. Prior to this research, program planners had speculated that parents without a Western educational background would not accept a learning model that was not based on the central leadership role of a teacher or an expert. It was argued that these group participants 64 would expect to have an "expert" teaching them the "right way" to parent and without this approach parents would not feel they had learned anything. This notion was shown to be incorrect as parents in these focus groups expressed strongly positive views on the effectiveness of the learner-centred program model. Focus group participants were asked to reflect on the effect of the program structure on their experiences in the program and to consider whether their Nobody's Perfect group had been 'facilitated" or "taught." Parents' descriptions of the group sessions indicated that delivery of Nobody's Perfect adhered very closely to the program guidelines. Of the responses to this question, 85% were elaborate and described the atmosphere of Nobody's Perfect as a program where parents learned and shared with other parents and with the facilitator. Many parents were able to identify the specific aspects of the model that they appreciated: I liked the fact that in the first session we lay down the ground rules of what was acceptable behaviour. We had a framework that we were working from so we understood andfelt comfortable. We had a lot of chances to input to Nobody's Perfect and to address the issues we wanted to talk about. She explained what was happening. We were talking all the time and she listened. She never said you should. 65 I liked it because she was also a mother and she had a struggle coming here and starting over and raising a family. And I liked it that she shared that with us... She could give us some insight. This group sharing format gave us opportunities to learn from the book, our facilitator, and gain support from other members. What I liked the best about this program was that when we came here we all had a lot on our minds including the stresses of our home lives and when I came to the group I felt a sense of relief and support and there was no pressure here. And everyone was very understanding and supportive of everyone's feelings. We all talked openly about our problems and we discussed our kids openly and how we all differed in child rearing. Our ideas were shared with everyone and we all learned from all of the other parents. We were all very friendly with each other it was a very amicable, learning atmosphere. Some parents' comments created an understanding of a developmental process where, over time, parents became increasingly comfortable with the learner centred approach. For example, when participants reflected on their experiences with the learner centred approach, their comments were highly descriptive and enthusiastic toward the model but noted that it took some time to adapt to it: 66 At first I didn't want to talk about my child not behaving but after a while I talked about it and everyone helped me. It did take some time before we could feel comfortable to open ourselves up in sharing. Our facilitator did a great job in encouraging us to share our experiences freely. It is not too difficult to share our problems regarding our children. It took time to begin sharing husband and wife conflicts and in-law conflicts. We may be afraid of losing face. In most cases the discussion which followed this question was a thoughtful reflection of each individual's experience with the group process itself. Overohelmingly, parents were able to identify the facilitation techniques used by the facilitator to encourage sharing among participants and they were very positive about the outcomes of this process. At the same time, some parents expressed a desire for the program to invite special speakers who are "experts" on specific parenting topics. The ability of the focus group discussion to uncover both aspects of parents' reactions is indication of the usefulness of focus groups in learning about complex issues such as a particular model of program implementation. 67 Did The Nobody's Perfect Program Seem Appropriate To Your Culture? Since 1989 when Nobody's Perfect had first been used in Vancouver, the implementation of the Nobody's Perfect Program in languages other than English had been controversial. Some of the criticism of other-than-EngUsh implementation was probably related to a general reluctance to provide services in languages other than English (and perhaps French) based on the view that immigrants should learn English. However, public discussion of this issue centred around questions of the "cultural appropriateness" of the program and whether the program which had been developed for a certain "target group" could address the needs of parents from diverse cultural backgrounds, especially those parents with limited English speaking ability. Some argued that the program content included in the five parent books did not address the specific parenting experiences of newcomer parents. Others had said that the program was "too Western" and many immigrant parents would not like it. The information provided by program participants in the focus groups described in this study made an important contribution to this discussion. Oveiwhehningly, parents spoke positively of their experience in the Nobody's Perfect Program emphasizing the contribution the program had made to their ability to successfully parent their children in a Canadian context. Of the 56 responses to the question about the cultural appropriateness of the program, 48 responses were elaborately descriptive providing insight into the process of settlement into a new culture. Parents reflected deeply on their experiences with parenting in the Canadian cross-cultural environment and how this had been addressed in the Nobody's Perfect Program 68 Parents' discussions allowed planners to understand the complexity of the settlement process and to recognize the careful and thoughtful consideration immigrant parents give to cultural differences and similarities as they explore various parenting approaches. Parents indicated that much of the discussion in the parenting program had addressed similarities and differences in child rearing amongst various cultures. Within the "safe" environment of the Nobody's Perfect Program parents had discussed the complex issues of parenting and in the equally safe environment of the first language focus groups these same parents were able to describe their views of cultural differences in parenting practices. The following quotations provide some insight into parents' reflections: Our background and life style as Chinese is different in many ways from the Westerners. The Westerners are more focused on their children while the Chinese are more focused on our job. The Westerners oppose giving physical punishment toward children. Chinese culture is for physical punishment. But I think we should keep a balance. We should give them the right degree of discipline. We are more conservative. We are not used to verbally and physically expressing and showing our love to our children by telling them "I love you" or giving them a hug. We prefer to hide our feelings in our hearts. Hugging and telling our children "I love you " while they are young is ok. But it is kind of awkward to do so when they are older. Anyhow, I am getting more used to the hugging now. 69 In this book, there are some similarities with the Vietnamese culture in teaching children, but there is a dissimilarity such as training children to be independent. It is the good way but being a Vietnamese mother, I am always worried that my children do not have enough warmth andfood. Actually, in Canada, to let children be independent since they are young is a better way. There are some similarities with the Vietnamese culture. One is that we need to be good so children can follow our example. Also, we train children when they are young because when they are older we cannot train them. How Were The Nobody's Perfect Books Useful To You And What Suggestions Do You Have For Improvement? The Nobody's Perfect Program includes five books intended for use during the group sessions and by parents at home. The five books are entitled Body, Safety, Mind, Behaviour, and Parents. Each book is about fifty pages in length and consists of both written text and accompanying illustrations. Each of the five books addresses sub-topics related to the title of the book. For example, the Safety book includes sections related to accidents, childproofing the home, toy safety, car safety, road safety, safety outside the home and first aid. The Behaviour book has sections entitled, "Love and Spoiling," "How Can I Teach Them to Behave?," "Spanking" and "Solving Some Common Behaviour Problems." From parents' comments in focus group sessions, program planners learned that parents greatly appreciated 70 the information contained in the books and used this information within their families. Planners also learned, however, that the usefulness of the books was significantly tempered by the inability of most participants to read English. The use of the program books by participants both during the program itself as well as at home was discussed at length in all of the focus groups. In total, 88 statements were made concerning what parents had learned from the books; 90% of these statements were elaborate descriptions. Regarding the use of the books during the Nobody's Perfect Program sessions, participants indicated that the degree to which the books were used depended primarily upon the group facilitator. Where the books had been used extensively, it was because the facilitator had provided either written or verbal translation of various sections of the books. In other groups facilitators had made verbal reference to information which they said was in one of the books. Use of the books at home was much more common. Some parents indicated that they looked in the books for information when they had a concern or a problem Many parents said they had tried to read the books at home, but often said they could only partially understand the text. Parents reported reading or 'looking at" the books with their children primarily to teach children how to behave. Some parents reported that they looked at the illustrations and guessed at the message being communicated. Some reported discussing these "guesses" with their children. The following quotations are examples of parents' comments: 71 I don't know English but I imagined and sometimes guessed what is in the books. We use the books with our children. Both parents and children enjoy them very much. We read, then put the books aside for a while and pick them up again. The pictures are very attractive and helpful. Just by looking at the pictures the children can understand the message. I'm trying to use the Mind book but I'm having trouble with the English. I've been looking for ways to help my daughter when other children take advantage of her at school. I think this book is useful to me even though I can understand only about 50% of the book. I have almost finished reading all of them except the safety book. This one I do not understand clearly. Our English level is not very high. It's not easy to understand the messages in the books. Even though we try to find out the meaning from the dictionary it takes time and sometimes we don't know which definition to pick. There are times when we were frustrated and lost interest in continuing reading. 72 I have not learned too much from the book. The book is not goodfor people who do not know how to read in English. Also, for people whose English level is low, there is lots of new vocabulary that needs to be looked in the dictionary; it will take lots of time to do it. The difficulties parents had accessing information from the books was important information for planners and policy makers. The realization that the program content might well be incorrectly interpreted by parents who could not read English and were dependent on the illustrations alarmed Health Canada officials. A specific example of this is the use of the terms "sorry" and "safe" with accompanying illustrations in the Safety book. Various typical situations are illustrated with one illustration showing an "unsafe" situation and the other a "safe" situation. However, without an understanding of the written words there could easily be confusion regarding the safety of specific situations. As well, it was realized that most of the illustrations were adequate when accompanied by the written text but could not take the place of the text in communicating the intended meaning. Almost all participants in the focus groups identified translation of the Nobody's Perfect parent books as the most critical need for change in the Nobody's Perfect Program Without translation of the books, parents felt they were not receiving the full benefit of the program A few parents used the simple, yet meaningful imperative, "translate the books" to express this idea. Most parents, however, used elaborate statements to explain why translation was needed and how translation of the books would benefit their learning. Parents provided very articulate 73 and convincing arguments in support of translation of the program books to increase the effectiveness of the program As a result, funding was provided to translate all five Nobody's Perfect books into Chinese, Punjabi, Spanish and Vietnamese and to distribute the books in translation to Nobody's Perfect groups as needed. The following provides examples of parents' comments about the language of the Nobody's Perfect books: The books are a resource but they are difficult because of the language. We need the information in our language to help us take care of our children. I would like the book translated because my English level is not enough to understand the book. I can get the information in the group, but not from the book. I only can see the pictures. Translate the books to meet the needs of those who are poor in English. When we use the English version we are just like little kids who have to guess the meaning by reading the pictures. The books have to be translated. They were made for Canadians but we need them now. I take the books home and I can't understand them. 74 Translate one book at a time if it is too expensive to do all of them. It may take quite a while before the whole set can be finished. But it is still worth it. It can benefit other mothers in the future. What Ideas Do You Have For Improving The Nobody's Perfect Program? Experienced planners of the Nobody's Perfect Program had learned over time that English speaking program participants typically suggested three improvements to the program Focus group discussions analyzed for this study suggested that non English speaking parents, regardless of their linguistic or cultural background, made the same suggestions for program improvement: programs with more sessions, opportunities to continue to meet periodically, and follow-up programs for parents of older children. In addition to the suggestions for translation of the Nobody's Perfect Program books, many parents in the non English speaking focus groups suggested the program be promoted through advertising in the media of specific cultural communities. The following comments from parents in the focus groups indicated their appreciation of the program and their desire to have the program extended and promoted: There should be more programs like this one and the length of these programs should be longer. There should also be a follow up group after the program finishes. 75 We would have liked to have gotten deeper into some topics but there was not enough time. There should be a group for parents of older aged kids and teenagers. We are worried about raising teenagers in this culture. The culture's influence is very strong and we want to be ready for what is coming. I hope that there will be a weekend or evening Nobody's Perfect program for working mothers for I will soon be going back to work after my maternity leave. I would like to take the program again in the future. I was ignorant of how or what to feed my baby. Through Nobody's Perfect I am doing fine now. This is an excellent program. It should be widely promoted. There are many other parents who need this kind of information and education. Especially those mothers who have to work full time, come home and take care of the family. The stress they face is tremendous. The Nobody's Perfect program should be advertised in the Vietnamese television, and be emphasized that it is in Vietnamese language. Also, the phone number should be given. 76 CHAPTER FIVE SUMMARY AND CONCLUSIONS Involvement of Non English Speaking Program Participants in Program Evaluation The purpose of this research was to determine the utility of focus groups as a program evaluation methodology with non English speaking program participants. A case study approach was used to study six focus groups which were conducted in the context of a program evaluation of the Nobody's Perfect Program conducted in Vancouver, Canada (Ritch & McLaren, 1994). These focus groups were implemented with non English speaking program participants to involve participants as stakeholders in the program evaluation process, to enable program planners and policy makers to learn about the experiences of non English speaking participants in the Nobody's Perfect Program and to improve the program based on the suggestions of those program participants. The English speaking researcher worked with bilingual, bicultural focus group facilitators in a "co-researcher" relationship prior to conducting the groups to establish a shared understanding of the focus group questions and to ensure consistent wording of the questions that would enable participants to contribute to the group conversation. Six focus groups were implemented, two each in Chinese and Spanish and one each in Punjabi and Vietnamese. An average often program participants attended each of the two hour focus group sessions. On-site child care was provided with caregivers who spoke the language of the parents attending. Each focus group was audio 77 recorded and the researcher was an observer at all sessions. Audio tapes of the sessions were transcribed and translated into English by bilingual, bicultural translators who were familiar with the content area of parenting programs. This familiarity with the language and approach of the Nobody's Perfect Program contributed to the production of meaningful transcripts in English. These transcripts were then analyzed by the researcher to determine whether focus groups in participants' first language had produced new and useful information for program planners and policy makers regarding the experiences of non English speaking Nobody's Perfect Program participants. Much of the information that was gathered through these focus group discussions was "hew" information for program planners and policy makers and provided suggestions for improving the effectiveness of program delivery of Nobody's Perfect to non English speaking parents. Information obtained through the focus groups conducted in this study regarding the use of the program books by non English speaking program participants is a case in point. Where previously the developer and publisher of the books, Health Canada, had been aware for some time that both staff and program participants had suggested translation of the books into several languages, it was the realization stemming from these focus group discussions that some information in the books was actually being misinterpreted by non English speaking readers that prompted officials to take immediate action to translate the books into four additional languages. The information gathered from these focus groups and the resulting policy and program changes supports the notion of the utility of focus groups as a program evaluation method with non 78 English speaking program participants. However, given the absence of information in the literature related to the use of focus groups with "special" populations, more research is needed to understand the process more completely. Jarrett (1993) suggests that the existing literature on focus group interviewing is useful for focus group implementation generally but that further work is needed to understand focus group interviewing with low-income and/or ethnic and racial minorities particularly addressing issues such as group composition, facihtation, advantages and limitations. The research for the study reported in this thesis also suggests the need to more fully understand the group discussion process itself within a specific cultural context. The focus groups described in this study were most certainly influenced by the fact that all program participants had completed the Nobody's Perfect Program and had become familiar with a group discussion format to express and exchange ideas. At the same time, regardless of their readiness to participate in such a group process, their participation in the Nobody's Perfect Program evaluation would not have been possible without the opportunity to provide feedback in their first language. Through these focus groups conducted in their first language, participants were able to participate as program stakeholders and to describe their experience in the program And although parents were not directly asked to describe changes in their parenting resulting from the program, many parents did suggest ways their everyday lives were affected by their participation in the program This information would have been inaccessible if the "first language focus group" methodology had not been utilized. 79 The recent direction in social services towards greater user involvement in program planning has had significant implications for program evaluation including how and from whom information is gathered (Croft & Beresford, 1990) . Conducting focus groups in participants' preferred language is a response to the need to include diverse stakeholders in program evaluation. It is a call for the application of methods which facilitate the meaningful participation of stakeholders in program evaluation (Kruegar, 1994) and highUghts the importance of doing research with specific sub populations in order to explore their particular needs (Kirkrnan-Liff & Mondragon, 1991) . Where people feel that their language ability will not allow them to fully express themselves, they may feel disempowered to participate in the process. Inviting their input through methodologies that allow participants full expression through use of their first language assures equity of outcome for these individuals and thus the ability to influence decision-making processes that may affect then-own lives. Morgan (1997) suggests that we judge the utility of focus groups by whether they help us reach our research goals. The important information resulting from implementation and analysis of focus groups with non English speaking program participants in this study is evidence of the usefulness of this qualitative method in eliciting information from specific populations which are generally excluded from research samples. In this study of the utility of focus groups for non English speaking participants, the information obtained from participants was crucial in helping evaluators realize the overall goals of the evaluation of the Nobody's Perfect Program The involvement of these participants was also important in that it gave these groups the opportunity to provide input into an evaluation that would affect them and to exercise some control on behalf of their own 80 interests (Guba & Lincoln, 1989). Through meaningful participation in the evaluation process, non English speaking parents can begin to take on the stakeholder role in their own services. Limitations of Focus Groups The primary limitation of focus groups discussed in the literature is that of the lack of generalizabihty of the information gathered. In quantitative research, sampling methods are used to ensure generalizabihty; such generalizabihty is not possible with focus group methodology. Even if quantitative sampling techniques could be used to assemble groups, the variations present in any group mcluding how much individuals within the group talk and how they share views within the groups would always preclude generalizabihty. In addressing the issue of focus group generalizabihty, Krueger (1994) suggests that focus groups are a valuable and useful methodology for programs in community-based social services agencies specifically because within such organizations, considerable credibility is given to the experience of the individual. Also, where it is important to hear from program participants directly concerning the value of a particular program to them, there may be sufficient reason to sacrifice some precision in measurement with the hope that this will increase the usefulness of the findings to persons in and around the program (Shadish et al, 1991) . Another limitation cited for focus groups is that the people included in the focus group are generally those who are ready to voice their views publicly. Potentially valuable contributors such as people with hearing or speech problems, very young and very old individuals, and people intimidated by articulating their views in public are likely to be excluded (Basch, 1987). 81 A further limitation is the general lack of direction in the literature regarding analysis of the results of focus groups. Much of the Hterature concerning focus groups describes methods for conducting the focus groups, but there is little discussion of what to do with the results. In the absence of such direction, researchers generally follow qualitative analysis procedures. For focus group analysis vvithin a program evaluation context, researchers generally describe and analyze the responses to focus group questions in terms of the usefulness of the information they provide to program planners and policy makers in making decisions regarding program delivery. In addition, information that would help researchers analyze the group dynamic among focus group participants or the dynamic of the facilitator and the participants is generally absent from the hterature. While the hterature does somewhat address the capacity and indeed the strength of focus groups to build knowledge through "group think" the dynamics of the group process are not sufficiently described. The limited information about group process in English language focus groups contributes to the difficulty of conducting analysis of the group dynamics within other cultural and linguistic groups. While the analysis of the focus groups for this study made no attempt to compare or contrast the focus group discussions of diverse cultures or to draw inferences regarding the role of cultural background in either the focus group process or the evaluative information generated by the focus group discussion, it seems evident that culture is an important factor in aspects of the focus group discussion such as turn-taking, leading the discussion, and "triggers" for the topic or pace of the discussion. One example of the influence of culture was noted in the focus group discussion in 82 Vietnamese in the way in which participants seemed to be "triggered" by statements made by the facilitator. In this group the facilitator periodically summarized what recently had been said by participants and in this summary the facilitator often included an idea or topic which had not been previously mentioned by group members. Following the facilitator's remarks, this "new" topic was usually addressed by many of the participants. This pattern of conversation was congruent with cultural practice in the Vietnamese community where individuals who are held in high esteem are often emulated by others. The focus group facilitator would hold such a place of honour and respect and, therefore, participants would follow the facilitator's lead regarding topics for discussion. While this pattern may suggest to some analysts that the focus group may not be a useful method for program evaluation because participants are to some degree 'led" by the facilitator, this conclusion may be a niisunderstanding of the process in its cultural context. This example of the "triggering" process is only one example of aspects of cultural dynamics which are deeply embedded in the focus group process and which require further study so that focus groups may be utilized effectively with diverse populations for program evaluation purposes. Because it is not possible to draw definitive conclusions from this one study, cultural dynamics in these areas warrant further study in order to extend our understanding of the utility of this methodology for program evaluation. 83 Translation Processes The need for translation of research instruments and research data is a major factor which is cited as hindering the amount of research done with linguistic minorities and which has contributed to the reluctance to conduct research with non English speaking populations. The cost of translation and a lack of confidence in the reliability of translated instruments are the two most prevalent reasons cited for avoiding situations where translation of research instruments would be required. The strengths and weaknesses of several methods for translation are described in the literature. Clarke (1992) describes the analysis of qualitative studies as a process of attribution by the investigator of meaning or importance to the data. She raises the question of whether sets of data give rise to the same categories when interpreted by various investigators. Interpretation and the attribution of meaning is of special concern where analysis is being done using transcriptions which have been translated from one language into another. It is important that adequate care be taken to define and check terminology as far as possible (Clarke, 1992) to ensure that the speaker's intended meaning is translated accurately. Because of the need for sensitivity to culturally-specific terms, it is important that translators be familiar with the concepts that are inherent in the program being evaluated. For example the term "Nobody's Perfect" cannot easily be translated into the Chinese language because the notion that one might not strive for perfection is so incompatible with the values of this culture that an equivalent phrase simply does not exist. Translators used Chinese terminology that suggested ideas that were compatible with the intention of the program title and were understandable within Chinese culture. Careful attention to 84 definitions and checking of tenninology as well as back translation techniques or verifying translation through more than one translator, while costly, can serve as mechanisms to ensure consistent attribution of meaning to the data. Program Evaluation Standards Focus groups conducted with non English speaking participants of the Nobody's Perfect Program will contribute to the ability of evaluations to address the Program Evaluation Standards published by the Joint Committee of Standards (1994) . These standards call for evaluations which have utility, feasibility, propriety and accuracy. The utility standards are intended to ensure that the evaluation will serve the information needs of intended users (The Program Evaluation Standards, 1994). One of the aspects identified as contributing to this utility is the identification of stakeholders in the evaluation and attention to addressing their needs. As programs are developed to encourage participation of people from diverse cultural and linguistic backgrounds, program evaluation must develop methodologies to include stakeholders who are often omitted because their language and cultural differs from the program evaluator. As these programs are initiated and become more and more established in the community, the need to evaluate them becomes paramount. Clearly, evaluation designs and methodologies must be developed to include these stakeholders in program evaluation processes. Focus groups in program participants first language are one method which can be effectively utilized to include non English speaking stakeholders in program evaluation and to help researchers learn more about program delivery in diverse cultural groups. 85 BIBLIOGRAPHY Adler, L. L. (1977) . Issues in cross cultural research. Annals of the New York Academy of Sciences. Anderson, G. (1990) . Fundamentals of educational research. Falmer Press Teachers' Library Series: 1. 17-26 . New York: The Falmer Press. Baizerman, M., & Compton, D. (1992). From respondent and informant to consultant and participant: the evolution of a state agency policy evalaution. New Directions for Program Evalaution. Number 53, 5-17. Basch, CE. (1987) . Focus group interview: An underutilized research technique for improving theory and practice in health education. Health Education Quarterly. 14. 411-448. Berrien, F. K (1967) . Methodological and related problems in cross-cultural research. International Journal of Psychology 2. 33-43. Bornstein, M., TaL J., & Tamis-LeMonda, C. S. (1991) . Parenting in cross cultural perspectives: The United States, France, and Japan. In Marc H. Bornstein (Ed.). Cultural Approaches to Parenting. Hillsdale, NJ: Lawrence Erlbaum Boshier, K (1991) . Psychometric properties of the alternative form of the education participation scale. Adult Education Quarterly. Volume 41. Number 3. 150-167. Brislin, KW. (1970) . Back translation for cross cultural research. Journal of Cross-Cultural Psychology. 1(3). 185-216. Brislin, KW. (Ed) . (1976) . Translation: Applications and research. New York: Gardner Press. Brislin, K W. (1981) . Cross-cultural encounters, face-to-face interaction. New York: Pergamon Press. Brislin, RW.,& Yoshida, T. (Eds.) . (1994) . Improving intercultural interactions. Sage Publications. Brochu, C. (1992). Evaluation report on the Quebec pilot project introducing the program entitled "Y'a personne de parfait" ["Nobody's Perfect"]. Trois-Rivieres, Quebec: Department des sciences de 1'education, Universite du Quebec a Trois-Rivieres. 86 Casagrande, J. B. (1954) . The ends of translation. International Journal of American Linguists. Vol. 20 No. 4. 335-340. Clarke, L. (1992) . Qualitative research: Meaning and language. Journal of Advanced Nursing. 17. 243-252. Crick, M. (1976) . Explorations in language and meaning: Towards a semantic anthropology. London: Malaby Press. Croft, S., & Beresford, P. (1990) . Listening to the voice of the consumer: A new model for social services research. Convergence. XXITI(4). 62-70. Cross, P. K (1981) . Adults as Learners. San Francisco: Jossey-Bass. DangeL K F. & Polster, K A. (Eds.). (1985) . Parent Training. New York: The Guildford Press. Dembo, M.H., Sweitzer, M., & Lauritzen, P. (1985) . An evaluation of group parent education: Behavioral, PET, and Adlerian programs. Review of Educational Research. 55, 155-200. Epstein, J.L. (1986). Parent involvement: Implications for limited-EngUsh-proficient parents. In C. Simich-Dungeon (Ed.), Issues of parent involvement and literacy. Proceedings of the Symposium Held at Trinity College. Washington, DC. (ERIC Document Reproduction Service No. ED 275 208) Ervin, S. & Bower, R. T. (1953) . Translation problems in international surveys. Public Opinion Quarterly, Winter. 595-604. Gladstone. JR. (1972) . Language and culture. In H.B Allen & R.N. Campbell (Eds.), Teaching English as a second language (pp. 192-195). United States: McGraw-Hill. Guba, E. G., & Lincoln,Y. S. (1989) . Fourth generation evaluation. Newbury Park: Sage Publications. Heath, S. B. (1983). Ways with words. Language, life, and work in communities and classrooms. New York: Cambridge University Press. Hui, C.H., & Triandis, H.C. (1983) . Multistrategy approach to cross cultural research. Journal of Cross Cultural Psychology, 14(1), 65-83. Jarrett, R. L. (1993) . Focus group interviewing with low income minority populations: a research experience In David L. Morgan, (Ed.), Successful focus groups-Advancing the state of the art. ,(pp. 184-201). Newbury Park, CA: Sage Publications 87 Johnson, C, Beiser, M. & Krech, K H. (1991) . The delivery of parent education services to ethnocultural groups: review and recommendations. Ottawa: Health and Welfare Canada. Joint Committee of Standards (1994). The program evalaution standards. Thousand Oaks, CA: Sage Publications. Kkkman-Liff, B. & Mondragon, D. (1991). Language of interview: Relevance for research of southwest Hispanics. American Journal of Public Health. 1399-1404. KnodeL J. (1993). The design and analysis of focus group studies: A practical approach. In D.L. Morgan (Ed.), Successful Focus Groups: Advancing The State Of The Art (pp. 35-50, Newbury Park, CA: Sage Publications. Krueger, R. A. (1994) . Focus groups. A practical guide for applied research. Thousand Oaks, CA: Sage Publications. Lengua, L. J., Roosa, M. W., Schupak-Neuberg, E., Michaels, M. L., Berg, C. N., & Weschler, L. F. (1992) . Using focus groups to guide the development of a parenting program for difficult-to-reach, high risk families. Family Relations. 41. 163-168. Lightbown, P.M. & Spada, N. (1993) . How languages are learned. Oxford: Oxford University Press. Lincoln, Y.S. (1991) . The arts and sciences of program evaluation. Evaluation Practice. 12. 1-7. Lincoln, Y.S., & Guba, E.G. (1985) . Naturalistic Inquirv. Beverly Hills, CA: Sage Publications. Lofiand, J. & Lofland, L. H. (1995) . Analyzing social settings. A guide to qualitative observation and analysis. Belmont, CA: Wadsworth Pubhshing. Madison, A. (1992) . Primary inclusion of culturally diverse minority program participants in the evaluation process. New Etirections for Evaluation. Minority Issues in Program Evaluation. 53. 35-43. Martin, J.R. (1992) . Beyond exchange: Appraisal systems in English. Paper presented to Sydney Linguistics Circle, Macquarie University. Merriam, S. B. (1988). Case study research in education. San Francisco: Jossey-Bass. Mohan, B. A. (1986) . Language and content. Reading, MA: Addison-Wesley Pubhshing. 88 Mohan, B. A., & Schwab, M. (1997, June) . Supporting the voices of diversity: Immigrant students and the discourse of attitudes. Paper presented at the International conference on Multiculturalism and Minority Groups, The Hebrew University of Jerusalem. Morgan, D. L. (Ed.) . (1993). Successful focus groups. Advancing the state of the art. Newbury Park, CA: Sage Publications. Morgan, D. (1997) . Focus groups as qualitative research. Newbury Park, CA: Sage Publications. Morgan, D.L., & Krueger, KA. (1993). When to use focus groups and why. In D.L. Morgan (Ed.), Successful focus groups: Advancing the state of the art (pp. 3-19). Newbury Park, CA: Sage Publications. Morgan, D.L., & Spanish, M.T. (1984) . Focus groups: A new tool for qualitative research. Qualitative Sociology. 7. 253-270. Mullen P.D & Reynolds, K (1982) . The potential for grounded theory for health education research: Linking theory and practice. In J.G. Zapka (Ed): SOPHE heritage collection of health education monographic. Volume 3: Research and Evaluation in Health Education. Oakland, CA: Third Party Publishers. MunhalL P. L & Oiler, C.J. (1986) . Language and nursing research. In Patricia L Munhall & Carolyn J. Oiler (Eds.), Nursing research: A qualitative perspective. Norwalk, CT: Appleton-Century-Crofts. Patton, M. Q. (1990) . Qualitative Evaluation and Research Methods. (2nd ed.) . Newbury Park, CA: Sage Publications. Patton, M. Q. (1997) . Utilization focused evaluation. (3rd ed.) .Thousand Oaks, CA: Sage Publications. Pomerleau, A., Malcuit G., & Sabatier, C. (1991) . Child-rearing practices and parental beliefs in three cultural groups of Montreal: Quebecois, Vietnamese, Haitian. In Marc H Bornstein. (Ed.), Cultural approaches to parenting (pp. 45-67) . Hillsdale, NJ: Lawrence Erlbaum Prieto, AJ. (1992) . A method for translation of instruments to other languages. Adult Education Quarterly. Volume 43. Number 1. Prince, K, & Mombour, W. (1967) . A technique for improving linguistic equivalence in cross-cultural surveys. International Journal of Social Psychiatry 13. 229-237. Ritch, A, & McLaren, N. (1994). East Vancouver Nobody's Perfect ethnospecific and multicultural project final report. Vancouver, BC: Kiwassa Neighbourhood House. 89 Rivers and Associates. (1990) . The Nobody's Perfect Program: a process evaluation. Victoria, BC: Ministry of Health. Robinson W.P. (1972) Language and social behaviour. Penguin Education, Harmondsworth. Rossi, P. H & Freeman, H. E. (1993) . Evaluation: A systematic approach. Newbury Park, CA: Sage Publications. Shadish, W. R. Jr., Cook, T. D., & Leviton, L. C. (1991) . Foundations of program evaluation: Theories of practice. Newbury Park: Sage Publications. Sinaiko, W. H, & Brislin, R. W. (1973). Evaluating language translations: Experiments on three assessment methods. Jounal of Applied Psychology 57. 328-334. Slaughter, H. B. (1991). The participation of cultural informants on bilingual and cross-cultural evaluation teams. Evaluation Practice. 12. No. 2, 149-157. Stake, R. E. (1995) . The art of case study research. Thousand Oaks, CA: Sage Publications. Stansfield, C. W. (1996) . Content assessment in the native language. ERIC/AE Digest Series EDO-TM-96-02. Stevens, S. B. (1993) . Community Based Programs for a Multicultural Society. Winnipeg, Manitoba: Planned Parenthood Manitoba. Stewart, D.W. & Shamdasani, P.N. (1990). Focus groups: Theory and practice. Applied Social Research Methods Series. Vol. 20. Newbury Park, CA: Sage Publications. Thorne, S. (1994) . Secondary analysis in qualitative research: Issues and implications. In J.M. Morse (Ed.) , Critical issues in qualitative research methods (pp. 263-279) . Thousand Oaks, CA: Sage Pubhcations. VanderPlaat, M. (1988) . Nobody's Perfect: a process and impact report. Atlantic Regional Health Promotion Committee. Vaughn, S., Schumm, J., & Sinagub, J. (1996) . Focus group interviews in education and psychology. Thousand Oaks, CA: Sage Pubhcations. Weiss, H. B, & Jacobs, F. H (Eds.) . (1988) . Evaluating family programs. New York: Aldine De Gruyter. I 90 Werner, O., & Campbell, D.T. (1970) . Translating, working through interpretors, and the problem of decentering. In K NarolL & R. Cohen. (Eds), A handbook of method in cultural anthropology (pp 398-420). Yin, R. K (1994). Case study research: Design and methods. (2nd ed.) . Thousand Oaks, CA: Sage PubUcations. 


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics

Country Views Downloads
United States 23 0
Australia 15 0
China 8 27
France 5 0
Russia 4 0
Germany 3 5
Canada 2 0
Ukraine 1 0
City Views Downloads
Templestowe Lower 15 0
Unknown 15 19
Washington 12 0
Ashburn 5 0
Shenzhen 5 27
Beijing 3 0
Saint Petersburg 3 0
Arlington Heights 1 0
Mountain View 1 0
Los Angeles 1 0

{[{ mDataHeader[type] }]} {[{ month[type] }]} {[{ tData[type] }]}
Download Stats



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items