UBC Faculty Research and Publications

Evaluation of a community-based, family focused healthy weights initiative using the RE-AIM framework Jung, Mary E; Bourne, Jessica E; Gainforth, Heather L Jan 26, 2018

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12966_2017_Article_638.pdf [ 863.51kB ]
JSON: 52383-1.0363331.json
JSON-LD: 52383-1.0363331-ld.json
RDF/XML (Pretty): 52383-1.0363331-rdf.xml
RDF/JSON: 52383-1.0363331-rdf.json
Turtle: 52383-1.0363331-turtle.txt
N-Triples: 52383-1.0363331-rdf-ntriples.txt
Original Record: 52383-1.0363331-source.json
Full Text

Full Text

RESEARCH Open AccessEvaluation of a community-based, familyfocused healthy weights initiative using theRE-AIM frameworkMary E. Jung1* , Jessica E. Bourne2 and Heather L. Gainforth1AbstractBackground: Childhood overweight and obesity is a major public health concern. Community-based interventionshave the potential to reach caregivers and children. However, the overall health impact of these programs is rarelycomprehensively assessed. This study evaluated a physical activity and healthy eating family program (HealthyTogether; HT) using the RE-AIM framework.Methods: Ten sites implemented the 5-week program. Thirty-nine staff members and 277 program participants(126 caregivers [Mage = 35.6] and 151 children [Mage = 13]) participated in the evaluation. Each RE-AIM dimension wasassessed independently using a mixed-methods approach. Sources of data included archival records, interviews andsurveys. Effectiveness outcome variables were assessed at pre- and post-intervention and 6-monthfollow-up.Results: Reach: HT participants were almost entirely recruited from existing programs within sites. Effectiveness:Caregivers’ nutrition related efficacy beliefs increased following HT (ps < .03). Participation in HT was not associatedwith significant changes in physical activity or nutrition behaviour or perceived social support (ps > .05). Knowledgesurrounding healthy diets and physical activity increased in children and caregivers (ps < .05). Adoption: Thirty-fivepercent of sites approached to implement HT expressed interest. The 10 sites selected recruited existing staff membersto implement HT. Implementation: Program objectives were met 72.8% of the time and 71 adaptations were made. HTwas finance- and time-dependent. Maintenance: Two sites fully implemented HT in the follow-up year and 5 sitesincorporated aspects of HT into other programs.Conclusions: Working alongside organizations that develop community programs to conduct comprehensive,arms-length evaluations can systematically highlight areas of success and challenges. Overall HT represents a feasiblecommunity-based intervention; however further support is required in order to ensure the program is effective atpositively targeting the desired outcomes. As a result of this evaluation, modifications are currently being implementedto HT.Keywords: Program evaluation, Children, Healthy weight, Community, Evaluation of a community-based,Family focused healthy weights initiative using the RE-AIM framework* Correspondence: mary.jung@ubc.ca1School of Health and Exercise Sciences, Faculty of Health and SocialDevelopment, University of British Columbia, Kelowna, CanadaFull list of author information is available at the end of the article© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Jung et al. International Journal of Behavioral Nutrition and Physical Activity (2018) 15:13 DOI 10.1186/s12966-017-0638-0BackgroundChildhood overweight and obesity are related to numer-ous proximal and distal health problems including heartdisease, high blood pressure and diabetes [1–4], withevidence suggesting that children who are obese aremore likely to be obese in adulthood [5]. One in everyfour Canadian children is considered overweight orobese [6]. Given the economic burden of obesity inCanada is estimated to be $7.1 billion annually [7], child-hood overweight and obesity is a public health issue thatrequires urgent attention.Several social and environmental factors have beenassociated with an increased risk of childhood obesityincluding being of aboriginal or multi-ethnic descent [8],low-socioeconomic status [9] and residing in rural andremote locations [10]. Given that children have littlecontrol over the social and environmental situations inwhich they live, the community has been highlighted asan important context through which to promote obesityprevention initiatives [11, 12]. Community based initia-tives may be beneficial over school initiatives as they caninclude parents, engagement of whom is considered keyin tackling pediatric obesity [13, 14]. Results fromcommunity-based interventions show potential effective-ness, with positive changes reported in clinical andbehavioural outcomes at the individual level [15, 16].While effectiveness is critical, few community-basedintervention studies have examined intervention effect-iveness while simultaneously assessing external indica-tors of success including program implementation,sustainability and maintenance within the desired con-text. Examination of both internal and external indica-tors of success are vital for assessing the long-termpublic health impact of an intervention.The RE-AIM framework provides a systematic methodthrough which to assess the overall impact of an inter-vention taking into consideration both internal andexternal validity [17, 18]. The framework outlines fivedimensions to be asessed (Reach, Effectiveness, Adoption,Implementation and Maintenance) and has been used todesign, implement and evaluate health promotion initia-tives extensively [19–21]. Reporting on each of thesedimensions enables health professionals to compare find-ings across interventions and establish the receptivity andsustainability of a program, enabling informed decisionsabout future public health initiatives. Comprehensiveevaluation is also essential to establish the degree of im-pact that community designed and implemented pro-grams have on the targeted population and community.Finally, it enables evaluators to determine how and whyan intervention works, permitting future refinement.The purpose of the current study was to use theRE-AIM framework to comprehensively evaluate the firstiteration of Healthy Together, an education programdesigned for children and their caregivers, developed by acommunity organization.MethodsProgram overviewHealthy Together (HT) is a family centered education pro-gram developed by The Bridge Youth and Family Services(hereon in referred to as ‘The Bridge’) to promote healthyweights in children from vulnerable populations (i.e., rural,remote, northern, Aboriginal and multicultural communi-ties across Canada; see http://healthy-together.ca). The firstiteration of HT comprised of 3 age-based modules: HappyHealthy Beginnings (0-6 years; Module 1), Fun HealthyHabits (7-12 years; Module 2), My Life, My World, MyChoice (13-18 years; Module 3). Each module consisted offive weekly sessions lasting approximately 2.5 hours, in-corporating children and caregivers. Each session was de-signed to provide attendees with information, skills andexperiences to support families in making healthy food andactivity choices, and included 15–30 minutes of play-basedphysical activity, 30–60 minutes of group discussion and45–60 minutes of cooking and eating together. Caregiversand children were provided with take home sheets aftereach session to complement discussions. Implementationsites were encouraged to adapt the discussions and handoutmaterial based on the needs of their population. Ten orga-nizations from five regions across Canada implementedHT over 2 years (five sites per year) and took part in theprogram evaluation between October 2013 and June 2015.ParticipantsAll caregivers and children (7–18 years) who registeredfor HT were eligible to take part in the evaluation ofprogram effectiveness. Children aged 0–6 were notasked to complete any evaluation documentation. Thecommunity site director, HT program coordinator, andfacilitators were all invited to participate in the evalu-ation. Facilitators conducted each of the HT sessions;the HT program coordinator oversaw all logistical as-pects of running the program and the community dir-ector oversaw all financial and community programing.Each site had a trained onsite evaluator who obtainedwritten consent from all participants and assent fromcaregivers of children 7–16 years of age.DesignEach RE-AIM dimension was assessed independentlyusing a mixed-methods approach to provide aninsightful evaluation enabling greater validity of infer-ences [22]. A mixed-methods approach was used be-cause qualitative data can provide further and richerinsights into quantitative findings. In particular, we pri-marily used quantitative measures to examine outcomesand qualitative findings to examine processes. Table 1Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 2 of 16Table 1 RE-AIM measures and data sources used to obtain informationAssessment level Measures Data sources TimelineReach • Eligibility criteria • Interview data (Director)• Pre Survey items (Coordinator)Pre-intervention• # Children/families in area, served bycommunity center that meet that criteria• Statistics Canada• Interview data (Director)• Survey data (Coordinator)Pre-intervention• # Children/families registered for theprogram• Summary Forms (Coordinator) Post-intervention• # Exclusions • Survey data (Coordinator) Post-intervention• Recruitment strategies • Interview data (Director)• Survey data (Coordinator)Pre-intervention• Identification of facilitators and barriers torecruitment• Interview data (Director)• Survey data (Coordinator)Pre-interventionEffectiveness • Short-term attrition rates • Survey (Caregivers, Children) Baseline, post-intervention• Healthy eating and physical activityefficacy beliefs• Knowledge related to healthy eating,physical activity and screen timeguidelines.• Perceived social support for physicalactivity and healthy eating• Healthy eating, physical activity and screentime behavour• Children’s health related quality of life• Survey (Caregivers, Children) Baseline, post-intervention• Perception of impact/consequences(positive or negative)• Interview data (Facilitators and Directors) Post-interventionAdoptionSetting Level • Criteria for implementing HT • Documentation (The Bridge) Post-intervention• # of settings approached to implement• # of settings that expressed interest inimplementing HT• Num. of sites expressing interest that wereexcluded• Difference in settings between 1) thosethat expressed interest vs. no-interest. 2)exclusion vs. inclusionStaff Level • Exclusion criteria• # staff approached/applied to be partof HT• Degree to which staff participating in HTare representative of staff at the centre• Survey data (Directors) Pre-intervention• Barriers to staff participation • Survey data (Facilitators, Directors,Coordinators)Pre-interventionImplementation • Degree to which project objectiveswere met• Observations During-Intervention• Degree to which program activitieswere administered• # and type of adaptations made • After-session survey (Facilitators) During-intervention• Program attendance rate • Observations During-intervention• Barriers and facilitators to implementation • Interview (Facilitators) Post-intervention• Time cost of the intervention • After-session surveys (Facilitators) During-intervention• Financial costs of the intervention • Documentation (The Bridge) Post-InterventionJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 3 of 16provides an overview of the variables assessed undereach RE-AIM component, the data source used to assesseach variable, and the data collection timeline.ProcedureEthical approval for this study was granted by the Uni-versity of British Columbia research ethics board andthe Public Health Agency of Canada ethics committee.Prior to implementing HT all facilitators received 2-daysof program training from The Bridge. Training consistedof a) project orientation (including overviews of theBridge organization, budget information and the role ofevaluation), b) education on group facilitation, c) reviewof the program content and the facilitator manual and d)planning and role playing one program session in groupsof three. Facilitators were also educated on dealing withshame and trauma and general considerations whenworking with children-in-care. Simultaneously, theevaluation team provided 2-day training for staff mem-bers of the implementation sites who would be acting asonsite evaluators. Onsite evaluators were responsible forcollecting all survey data from participants and conduct-ing observations.MeasuresStaff surveysStaff surveys were used to gather demographic informa-tion and information pertaining to Reach, Adoption andMaintenance using a variety of open-ended, categoricaland likert-type questions. Directors completed surveys atbaseline (n = 9) and 1-year follow-up (n = 5), whilecoordinators completed surveys at baseline (n = 8), postintervention (n = 6) and at 1-year follow-up (n = 4).Each survey took approximately 20-minutes to complete(Additional file 1 provides information on the data sourceused to assess each of the RE-AIM dimensions and thetime of data collection). Facilitators completed surveys atbaseline (n = 24) and after each program session, whichtook approximately 10-minutes to complete.Child and caregiver surveysSurveys were developed by the researchers to map ontothe components of the HT program manual, and wereadministered to children and caregivers at pre-, post-,and 6-month follow-up. Demographic information wascollected at baseline. The child survey measured know-ledge, efficacy beliefs and behaviour in relation tohealthy eating, physical activity and screen time. Healthrelated quality of life and perceived social support forhealthy eating and physical activity were also measured.Cronbach’s alpha for composite measures within thechild survey ranged from .71 to .93 (see Additional file 2).All surveys were pilot tested with children (n = 12) to as-sess readability and receptiveness. The caregiver surveyassessed knowledge, efficacy beliefs and behaviour in re-lation to health eating, physical activity and screen time.Perceived social support provided to children and care-givers’ reports of child health related quality of life wasalso assessed. Cronbach’s alpha for composite measureswithin the caregiver survey ranged from .57 to .84 (seeAdditional file 3). Where possible previously validatedmeasures were used. Where no vailidated measures wereavailable study specific measures were created andreviewed by all authors for content validity.InterviewsDirectors completed individual interviews at baseline(n = 9) and post intervention (n = 10). Baseline interviewsincluding questions pertaining to HT reach and staff leveladoption, while post interviews included questionspertaining to the consequences of running the HT(effectiveness). Two in-depth group interviews wereTable 1 RE-AIM measures and data sources used to obtain information (Continued)Assessment level Measures Data sources TimelineMaintenanceIndividual Level • Long term study attrition • Survey (Caregivers, Children) Pre, 6-month follow-up• Healthy eating and physical activity efficacybeliefs• Knowledge related to healthy eating,physical activity and screen timeguidelines.• Perceived social support for physicalactivity and healthy eating• Healthy eating, physical activity and screentime behaviour• Children’s health related quality of life• Survey (Caregivers, Children) Pre, 6-month follow-upSetting Level • # of sites running Healthy Together • Surveys (Directors, Coordinators) 1-year follow-up surveyand email• Adaptations of the program – retainedelements• Reasons for lack of implementationJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 4 of 16conducted with facilitators post intervention (n = 19). Onegroup interview was conducted in year 1 and the secondin year 2. Questions assessed the dimensions of programeffectiveness and implementation and lasted 1-h.ObservationsOnsite evaluators observed all program sessions withintheir community site. Evaluators rated whether facilita-tors met session objectives (as laid out in the programmanual), presented and ran each activity, and whetherfacilitators adapted or added any material to the sessionusing a ‘Yes’ or ‘No’ response scale.DocumentationInternal program documents describing the process ofcommunity site involvement in HT was obtained fromthe Bridge to assess setting level adoption as well asfinancial information pertaining to the implementationof the program. Summary forms completed by eachcommunity site were obtained to assess program reach.Archival records included census data from StatisticsCanada and school statistics. To gain an understandingof program maintenance, emails were sent to directorsand coordinators of sites that did not return their 1-yearfollow-up survey.Data analysisChild and caregiver survey data was analyzed using SPSS(v24, 2016). Descriptive statistics were calculated for allstudy variables. Data was screened for outliers usingbox-plots and assumptions of normality were assessedusing the Shapiro-Wilk test of normality using differencescores. Partial missing data (i.e., less than 50% of a scale)were replaced using a series mean [23]. If differencescores were not normally distributed, non-parametricWilcoxon sign-test was conducted instead of a paired-sample t-test. Independent sample t-tests and chi-squared analyses were conducted to examine differencesin demographics and outcome variables between partici-pants that dropped out of the program and those thatremained at post and at 6-month follow-up. Effective-ness and individual level maintenance were analyzedusing a series of paired-samples t-tests, McNemar chi-squared or Wilcoxin sign-tests. Effect sizes werecalculated using odds ratios for McNemars chi-squared,r for Wilcoxon sign-test (0.1, 0.3, 0.5 represent a small,medium and large effect respectively; [24]) and cohen’s dfor paired sample t-tests (0.2, 0.5, 0.8 represent small,medium and large effect respectively; [24]). Data wereanalyzed collectively for all implementation sites andwhere possible individually by site (i.e., sample size per-mitting). See Additional file 4 for a list of sites for whichindividual site analyses was conducted for caregivers andchildren. Significance was set a p ≤ .05.To provide a further insights into processes that mayhave influenced quantitative findings, interviews andqualitative survey data were deductively analyzed usingthe RE-AIM framework [25]. All interviews were tran-scribed verbatim. Qualitative survey data was enteredinto an excel sheet. Two coders independantly per-formed a content analysis on both sets of data using thedomains of RE-AIM. The coders familiarized themselveswith the data by carefully reading the transcripts andsurvey responses. They deductively coded the data usingthe criteria for each of the five RE-AIM dimensions(Table 1). Any discrepancies were resolved through dis-cussion. Coded data was then reviewed by the researchteam to extract illustrative quotations that providedfurther insight into quantitative findings.ResultsReachTwo of the 10-implementation sites reported specific in-clusion criteria, beyond the age requirements specifiedby the modules (see Additional file 5 for data used to de-termine program reach). At one site participants wererequired to be refugees or new immigrants to Canada,while at the other children had to be attending a specificschool. Despite no other explicit eligibility criteria allsites targeted HT recruitment at specific populations in-cluding families of children at risk, families of Aboriginaldescent, women and new immigrants, and refugees toCanada. Based on the specified eligibility criteria it wasestimated that approximately 73,368 children were eli-gible to participate in HT across the 8 sites. Reasonableestimates of the potential eligible population were notavailable for two sites. A total of 223 caregivers and 398children registered for HT at 10 sites. At the 8 sites withpopulation estimates available 330 children registered,suggesting that the program reached approximately .45%of the potential target population. Fifty six caregiversand children were excluded from two sites for module 1due to lack of space. One family was excluded from theprogram as they did not drive and the organization wasunable to provide transportation. Of those that regis-tered for the program 190 individuals self-identified asAboriginal (31% of all registrants), 56 as immigrants(9%) and 42 children were classified as in care of thegovernment (11% of child registrants).RecruitmentThe primary recruitment strategies were verbal presen-tations and communication with a) existing programsoffered at the site, b) community partners and c) localschools and school boards. Three sites created HT infor-mation pamphlets and offered information sessions withincentives to encourage attendance. Four sites workedwith local schools to recruit and offered HT as part ofJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 5 of 16the school curriculum or during school time. For mod-ule 1 and 2 the majority of participants were recruitedfrom other programs being offered at the sites.The most commonly cited barrier to recruitment was alack of connection to the population of interest, particu-larly the 13–18 year olds and a desire from this age groupto participate independently of caregivers. As such, fivesites ran module 3 without caregivers. A lack of trust re-garding new programs with also highlighted as a barrier torecruitment. Directors explained that families with chil-dren of Aboriginal descent need extensive informationabout a program prior to participation to understand themotivation of the program and potential consequences ofparticipation. Two sites stated that transportation was anissue for recruitment and that the cost of providing trans-portation for participants was limiting.The most commonly cited enabler to recruitment wasthe presence of pre-existing programs within the sitefrom which to invite individuals to participate. Inaddition, working closely with community partnershelped to build the trust of participants and to recruitpopulations that the sites did not previously serve.EffectivenessStudy attritionA total of 126 caregivers completed the baseline survey(see Fig. 1 for flow of participants through the evaluationand Additional file 6 for caregiver demographic statis-tics). Of these caregivers 71 completed the post survey(43.65% attrition) and 38 completed 6-month follow-up(69.84% attrition). There were no differences in age, sex,ethnicity of adult or child or marital status between par-ticipants who completed the post survey and those thatdid not (ps > .05). There was a significant difference ineducation level (χ2(5) = 12.12, p = .03) such that care-givers who completed the post survey had a higher levelFig. 1 Flow of program participants through the evaluation. Percent represents participant retention in the evaluationJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 6 of 16of education than those that did not. There were nosignificant differences in outcome variables pre-intervention between those that completed the post sur-vey and those that did not (ps > .05).At 6-month follow-up there was no difference in age,ethnicity of adult or child, marital status or level of edu-cation. There was a significant difference in sex betweenthose that completed the survey and those that did not,χ2(1) = 4.13, p = .04. Females were 3.56 times more likelyto complete the 6-month follow up than males. Therewere no significant differences in outcome variables pre-intervention between those that completed the 6-monthfollow-up survey and those that did not (ps > .05).A total of 151 children completed the baseline survey(Mage = 13(±3); 57% female). Of these, 80 completed thepost survey (47% attrition) and 46 completed 6-monthfollow-up (70% attrition). No differences in sex, age orany outcome variables were found between those thatcompleted the post survey and those that did not, orthose that completed the 6-month follow-up survey andthose that did not (ps > .05).Caregiver outcomesKnowledge There were no changes in knowledge ofphysical activity requirements for adults or children,fruit and vegetable consumption requirements for adultand children, or knowledge of screen time limits frompre to post-program, or from pre to 6-month follow-up (ps > .05; see Table 2 for pre, post and 6-monthfollow-up changes for each outcome variable). No differ-ences in knowledge were found within individual sites(ps > .05) (Table 2).Efficacy beliefs Immediately following HT, caregiverswere more confident that they could engage in healthyeating practices, t(64) = −2.37, p = .02, d = .32. This in-crease was maintained 6-month follow-up, t(31) = −2.25,p = .03, d = .47. Individual site analyses revealed thatcaregivers from site B showed increased confidence toengage in healthy eating practices immediately after HT,t(5) = 3.16, p = 0.25, d = .27. This increase was not main-tained at 6-month follow-up, though only 2 individualscompleted both time-points. There was a significantincrease in caregiver’s confidence to cook immediatelyfollowing HT, t(63) = −3.40, p = .001, d = .43, this increasewas maintained at 6-month follow-up, t(31) = −2.78,p = .01, d = .49. Individual analyses showed that care-givers at site C showed increased confidence to cook bothimmediately after HT and at 6-month follow-up (ps < ..05,ds > .57). Immediately after HT there was a significant in-crease in caregiver’s confidence to involve their child incooking, t(61) = −2.02, p = .05, d = .32. No change in confi-dence to engage children in cooking was found betweenpre-HT and 6-month follow-up or amongst sitesindividually (p = .50). There were no changes in confi-dence to engage in physical activity from pre- to post-program or from pre to 6-month follow-up (ps > .05).However, individual site analyses revealed a significantincrease in confidence to engage in physical activityfor sites B and J from pre-post HT, t(5) = −2.93,p = .03, d = .44 and t(11) = −3.541, p = .01, d = 1.04respectively. Site J appeared to marginally maintainthis increase at 6-month follow-up, t(11) = −2.115,p = .06, d = .55.Behaviour Caregivers self-reported shopping practicesdid not change post HT or at 6-month follow-upcompared to pre HT, nor did the availability of healthyfood in the house (ps > .05). There was, a significant de-crease in the availability of unhealthy food immediatelyafter the program (t(54) = 2.37, p = .02, d = .31) and at6-month follow-up (t(27) = 2.79, p = .01, d = .53) com-pared to pre-program. Analyses of individual sites revealeda significant increase in healthy shopping practices preHTand at 6-month follow-up for sites D and I (ps < .05,ds > .49). Site A showed a decrease in the availability of un-healthy food immediately post program, t(7) = 2.78, p = .03,d = .67. Maintenance statistics were unavailable due to thelimited sample size. While Site J showed a decrease in theavailability of unhealthy food from before the program to6-month follow-up, t(8) = 2.89, p = .02, d = .57. Caregiversreported no changes in their children’s fruit and vegetableintake, their personal fruit and vegetable intake or theirpersonal breakfast consumption after HT (ps > .05). Nochanges in the frequency of consuming an evening mealwith their child were reported following HT (ps > .05). Nochanges were found in caregiver’s reports of children’sphysical activity behaviour or screen time hours outside ofschool hours or in caregiver’s personal physical activitypost HT or at 6-month follow-up compared to pre-intervention (ps > .05). Individual analyses revealed thatcaregivers in site A reported a decrease in children’s phys-ical activity from before (Mdn= 7) compared to after theprogram (Mdn= 4), Z = −2.06, p = .04, r = .73.Social support There were no changes in parental socialsupport provided for physical activity or healthy eatingimmediately after HT or at 6-month follow-up com-pared to baseline (ps > .05). However, site A showed asignificant decrease in social support for physical activityimmediately following the program, t(6) = 2.68, p = .04,d = .78. Insufficient sample size eluded examination ofsocial support for physical activity at 6-month follow-upfor site A.Health related quality of life No changes were re-ported in parental reports of children’s’s total health re-lated quality of life after HT or at 6-month follow-upJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 7 of 16compared to pre-intervention (ps > .05). No differencesin health related quality of life were found within sites.Children outcomesKnowledge There were no short (post-program) orlong-term (6-month) changes in children’s knowledge ofthe daily fruit and vegetable requirements or sugarydrinks (ps > .05). Knowledge of the recommended num-ber of minutes of daily physical activity increased frompre- to post-program, χ2(1) = 4.65, p = .03, OR = .21.This increase was not seen between pre-program and6-month follow-up (p = .18). Knowledge of screen timerecommendations was not changed (ps > .05; see Table 3for pre, post and 6-month follow-up changes for each out-come variable). No differences in knowledge were foundwithin individual sites (ps > .05) (Table 3).Efficacy beliefs Immediately after HT and at 6-monthfollow-up children reported no changes in their confi-dence to engage in physical activity compared to base-line (ps > .05). Individual site analyses revealed thatchildren at site B were more confident to engage inTable 2 Changes in outcome variables for caregiversMeasure Pre-Intervention Post-Intervention 6-month follow-upM SD M SD n p es M SD n p esKnowledgeaAdult daily fruit and vegetable servings 9.5 17.6 65 .15 .52 16.2 35 1.0 .16Child daily fruit and vegetables servings 20.8 42.9 25 .18 2.22 47.1 14 .22 .42Sugary drinks 78.4 74.3 67 .58 .11 91.7 34 .69 .29Adult minutes of PA per day 44.3 36.3 67 .15 .29 48.6 36 1.0 .25Child minutes of PA per day 41.7 67.6 25 .13 .08 58.8 15 1.0 .25Adult days of PA per week 50 50.7 68 .29 .21 45.9 36 .39 .24Child days of PA per week 46.6 49.3 67 .83 .24 56.8 36 .82 1Child screen time limits 26.7 37.7 68 .50 .20 41.7 35 .77 .31Efficacy BeliefsAdult healthy eating efficacy (1–5) 2.65 1.06 3.07 0.91 65 .02* .32 3.10 .93 32 .03* .47Cooking efficacy (0–100) 82.15 19.37 88.44 14.52 64 .001* .43 88.57 12.30 32 .01* .49Efficacy to include child in cooking (1–5) 3.35 1.05 3.66 .90 62 .05* .32 3.45 .98 30 .50 .12Adult physical activity efficacy 55.36 24.64 57.87 16.13 42 .10 .26 55.15 18.11 20 .44 .18BehaviourEat breakfastb (0–6) 4.00 4.00 61 .68 −.05 4.00 32 .89 .02Caregiver fruit and vegetable consumptionb (0–6) 3.50 4.00 60 .57 .07 4.00 31 .67 −.07Eat evening meal with childb (0–6) 5.00 5.00 56 .28 .15 5.00 29 .79 .05Shopping practices (0–4) 2.40 .92 2.50 .68 55 .72 .05 2.57 .82 25 .08 .37Child fruit and vegetable consumption (0–8) 4.45 2.08 4.35 1.74 54 .80 .04 4.97 1.96 26 .68 .08Healthy food availability (0–100) 66.82 15.62 70.12 13.98 55 .36 .12 67.12 16.30 29 .23 .23Unhealthy food availability (0–100) 41.42 16.11 39.66 17.80 55 .02* .31 33.76 19.92 28 .01* .53Child’s physical activity behaviourb (0–7) 5.00 4.00 55 .44 −.10 6.00 26 .35 .18Child’s sedentary behaviourb (0–6) 1.00 1.00 48 .08 −.25 1.00 23 .58 .12Caregiver physical activity behaviourb (minutes) 100.00 110.00 48 .31 .15 82.00 26 .15 −.28Social SupportPhysical activity 3.10 .92 2.91 .83 58 .16 .19 3.29 .88 58 .12 .31Healthy eating 4.08 1.00 4.28 .87 64 .32 .13 4.30 .87 31 .20 .23Healthy Related Quality of Life 71.64 16.52 74.66 9.42 20 .56 .14 75.62 16.75 14 .57 .24aValues reported in the M column represent percent correct based on all that answered the question. p values calculated using Exact McNemar’s chi-squared test,effect size calculated as the odds ratio. Odds ratios were calculated based on the chances of obtaining the correct answer following Healthy Together if youranswer was incorrect before the programbValues reported in the M column represent median score. p values calculated using Wilcoxon sign-test, effect size calculated as r*Significant difference between time points, p < .05Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 8 of 16physical activity at 6-month after HT compared to be-fore, t(2) = −5.05, p = .04, d = 2.89. Children were 0.33times more confident that they could consume fruitsand vegetables immediately after HT (Mdn = 4.0)compared to before HT (Mdn = 4.0), z = 2.70, p = .01.Individual analyses revealed that children specificallyfrom site C were 0.85 times more confident that theycould consume fruits and vegetables immediately afterHT (Mdn = 2.75) compared to before HT (Mdn = 3.5),z = −2.55, p = .01. No changes were reported in confi-dence to eat breakfast, choose healthy drinks or cook frompre- to post-HT or pre-HT to 6-month follow-up for thegroup as a whole or by site.Behaviour No changes were reported in children’s self-reported fruit and vegetable consumption, consumptionof unhealthy foods or pop intake, frequency of breakfastconsumption or frequency of assisting with cooking im-mediately after HT or at 6-month follow-up comparedto baseline for the whole group (ps > .05). Individually,children from site A showed an increase in theconsumption of unhealthy food from before to afterHT, t(8) = −2.32, p = .49, though the effect size wassmall (d = .12). Children at site I showed a decrease intheir consumption of unhealthy food from pre to postprogram, t(3) = 4.38, p = .02, d = 3.31. Children in Site Hshowed a reduction in unhealthy food consumption frombefore to 6-months after the program t(7) = 2.755, p = .03,d = 1.11 and a decrease in the consumption of pop intakefrom pre (Mdn = 3.5) to immediately after HT (Mdn = 3;z = −2.07, p = .04, r = −.69). Children did not show anychanges in screen time or physical activity behaviourfollowing HT as a whole or when assessed as individualsites (ps > .05).Social support No changes in perceived social supportfor physical activity or healthy eating were reported frompre- to post-program or from pre-program to 6-monthTable 3 Changes in outcome variables for children and youthMeasure Pre-Intervention Post-Intervention 6-month follow-upM SD M SD n p es M SD n p esKnowledgeaDaily fruit and vegetable requirement 15.1 (22) 22.8 (18) 75 .65 .25 23.8 (10) 42 .55 .33Sugary drinks (Module 3 only) 85.2 (69) 92.9 (39) 41 .45 .30 90.3 (28) 27 1.0 −.13Daily physical activity 43.8 (64) 55.8 (43) 76 .03* .21 62.2 (28) 43 .18 .20Screen time limitations 24.5 (34) 35.5 (27) 72 .52 .27 38.6 (17) 39 .09 .29EfficacyModerate-to-vigorous physical activity (0–100) 67.10 26.74 68.96 22.39 68 .08 −.21 67.90 25.29 33 .77 −.05Fruit and vegetable consumptionb (1–5) 3.50 3.50 65 .01* .33 4.0 29 .52 .12Eat breakfastb (1–5) 4.00 4.50 66 .39 .11 4.50 30 .34 −.17Choose healthy drinksb (1–5) 4.00 5.00 66 .56 .07 4.00 30 .82 −.04Cooking efficacy (1–5) 4.00 4.33 62 .45 .10 4.17 30 .78 .05BehaviourConsumption of fruits and vegetables (1–7) 4.06 1.64 3.88 1.63 44 .54 .09 4.08 1.77 27 .78 .05Consumption unhealthy food (1–7) 2.54 2.41 2.41 1.22 44 .98 .004 2.51 .89 26 .72 .07Consumption of regular popb (1–7) 2.00 2.00 52 .45 −.10 2.00 30 .08 −.32Eat breakfastb (1–5) 4.00 4.00 51 .93 .01 4.00 29 .10 .31Help cookb (1–5) 3.00 3.00 50 .44 .11 3.00 28 .28 .21Hours per day in front of screen (0–6) 5.43 3.69 4.96 3.45 54 .58 .08 5.45 2.76 29 .67 .08Num of days spent in MVPA 30 + mins 2.00 3.00 53 .78 .04 3.00 28 .89 .03Social Support (0–4)Physical activity 2.45 1.10 2.63 1.22 59 1.00 0 2.60 1.26 28 .88 .03Healthy eating 3.38 1.22 3.56 1.34 57 .29 .14 3.37 1.22 29 .49 .13Health Related Quality of Life (0–100) 71.12 15.82 73.16 16.82 56 .08 .21 73.42 15.31 29 .36 .15a Values reported in the M column represent percent correct based on all that answered the question. The number of responses in provided in parenthesis. p valuescalculated using Exact McNemar’s chi-squared test, effect size calculated as the odds ratio. Odds ratios were calculated based on the chances of obtaining the correctanswer following Healthy Together if your answer was incorrect before the programbValues reported in the M column represent median score. p values calculated using Wilcoxon sign-test, effect size calculated as r*Significant difference between time points, p < .05Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 9 of 16follow-up (ps > .05). Children from site A reported an in-crease in perceived social support for both physical ac-tivity and healthy eating immediately after the program,t(12) = −2.28, p = .041, d = .35 and t(12) = −2.28, p = .04,d = .30 respectively. The sample size did not permit ana-lyses of site A at 6-month follow-up.Health related quality of life Total health related qual-ity of life did not change between pre- and post-programor pre-program and 6-month follow-up for the group oras individual sites (ps > .05).Director and facilitator feedbackFacilitators believed that HT was well received byparticipants. One facilitator commented after session 3:“I really feel this program is making a difference to thepeople involved. Two families walked last week insteadof driving. They are trying foods they would have nevertrying and enjoyed them. Therefore, I see this programas a success already”. Another commented after the finalsession, “Families talked about changes they have madesince starting the HT program. One mom said she evenstarted making a shopping list and she said she neverdoes lists for anything”.Facilitators believed that HT helped strengthen familialrelationships and develop new relationships with individ-uals in the program. For example, one facilitator com-mented: “Oh, I thought it was great. Yes, definitely,having the mom and the dad and the kid come in andparticipate and have fun. With the teenagers it was likethe moms and the daughters and it was so nice to seethem like sit down, cook together. One of the girls shesaid like 'My mom always cooks I never cook' so hermom was like 'go, go, try it' so she cooked her first mealthere. It was an amazing program”. While another facili-tator stated; “I think it was a huge impact not only forthe physical benefits but also the social benefits. Theseare newcomers [to the country] and they got to meetwith other newcomers. They got to visit grocery stores.They got to do different activities that I think werepurely beneficial”. Involving both the caregiver and childin the program was perceived as the biggest benefit ofthe program with one facilitator stating; “Families arelearning to work together and be able to know andunderstand that if you want to be healthy you got to dodifferent things, it's a lifestyle change”.Directors felt that the program had a positive impacton participants while they were attending but were un-sure of the long-term impact of HT due to the length ofthe program; “I think this is a great program with a lotof potential. However, I feel that long-term change willrequire a longer time period. If this program was part ofa curriculum that took place over the school year, then Ifeel that there would be a better chance for long termchange”. This feeling was echoed by the facilitators whofelt that the program might be more effective if it waslonger, with 10 sessions being repeatedly mentioned as apotentially viable number.AdoptionSetting levelOne hundred community sites were approached to par-ticipate in the implementation of HT for which therewere 10 funded positions available over 2 years. Of the100 sites that were approached to participate 35reported being interested in implementing HT. It isunclear how these 35 sites distinctly differed from the 65sites that were not interested in participating in theimplementation of HT.Of the 35 sites that expressed interest in HT severalsites were unable to participate due to either limited ad-ministration and personnel capacity at the time of com-mitment or they felt that the group learning model wasnot feasible for implementation at their site (i.e., hospital,day care centre). Of those sites that could commit toimplementing the program, 10 were selected by TheBridge to include a representation of diverse communitysettings for program implementation. This included a)children in care, b) children with special needs, c) familiesof Aboriginal, Inuit, and/or Metis descent, d) immigrantsand refugees, e) families living in rural/remote, areas, f )families living in northern communities, and g) familieswith low income.Staff levelData on staff recruitment was available from 9 of the 10sites (see Additional file 7 for staff adoption data by site).Seven of the nine sites utilized staff already working forthe organization to implement HT. This was possible asthe employees were part-time and wanted additionalhours. In addition, implementing HT was not a full-timeposition and coordinators felt it would have been a chal-lenge to attract staff from outside the organization. Ofthese 7 sites, two approached specific staff members tobe part of HT while the other five encouraged interestedstaff members to apply. The remaining two sites hiredindividuals from outside of the organization to imple-ment HT, as the workload of their current staff was toohigh to ask them to implement HT in addition to theirother roles.Four of the nine sites had staff members within theirorganization who were not interested in implementingHT. These individuals were not interested in being partof HT as they were already employed full-time as coun-sellors or the program did not fit their qualifications(i.e., they did not work with children) or area of interest.Three of the 10 site directors felt that their facilitatorswere typical of the staff working at their site. One siteJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 10 of 16felt that their facilitators were those that enjoyed group-based work, which was not common within their childwelfare agency. Healthy Together facilitators had moreexperience working with children and facilitating groupsthan the other organization staff members and had moreinterest in general health. One director commented;“The staff who are involved are more curious and posi-tive and adventurous - all interested participants have astake in being healthy”. The majority of facilitators re-ported that they had been involved in implementing nu-trition and/or healthy eating programs in the past, whileonly two facilitators reported having implemented anyphysical activity related programs in the past. Staffdemographics are shown in Additional file 8 as a groupand in Additional file 9 facilitators demographics areprovided by site. Sites D, H and J had at least onefacilitator with a post-graduate degree, while site Dfacilitators identified as being from a range of ethnicbackgrounds including south and southeast Asian,Filipino, Black and Latin American. There appears to bea link between the demographic of staff or theirexperience and individual site effectiveness, although datais limited. In total 19 volunteers assisted with the HT pro-gram, contributing a total of 269.5 h of their time.ImplementationDegree to which program was implemented as per programmanualOn average, program objectives were met 72.8% of thetime across all modules (see Additional files 10, 11 and 12for specific module and session implementation out-comes). The cooking activity was conducted 99.3% of thetime while, other activities, including physical activity,were conducted less frequently (average 73.6%). Seven fa-cilitators felt that they implemented HT as outlined by theprogram manual, while facilitators at two sites felt thatthey were unable to implement the program as laid out bythe manual. These two sites felt that the material was notappropriate for their population due to either financial cir-cumstances or past trauma, therefore the content washeavily reduced. Due to bad weather one site had to canceltwo sessions for Module 3 and one session for Module 1.These sessions were not made up at a different time.Program adaptationsApproximately 71 adaptations were made to HT acrossall three modules and all sessions. Twenty four adapta-tions involved providing alternative handouts to thoseincluded in the manual and another 24 related to usingdifferent recipes to suit the group (i.e., stove top ban-nock). Eight adaptations related to changing the orderof the session activities. Three adaptations involvedomitted activities, specifically site E did not conductthe physical activity during one session and site B didnot cover the topic of family traditions due to fearsof emotional distress to participants. Ten adaptionsinvolved added components to HT, for example dem-onstrations of the sugar in drinks using sugar cubesby site C and bringing in a yoga instructor to teachthe children basic yoga poses at site E. Two adapta-tions were made that were unspecified.Barriers to implementationAll facilitators highlighted time constraints as the hard-est element to overcome when implementing HT. Facili-tators felt that the content of the program was tooextensive for the 2-hour time period in which imple-mentation was meant to occur. This restricted the facili-tators from delivering HT in its entirety or developingappropriate adaptations. One facilitator commented;“Time. It was really hard to fit in everything that youwere expected to fit in, in each module in the time thatwas allotted”. Due to the extenstive content particiapantswere reportedly overwhelmed with the amount of infor-mation; “I think sometimes though by the end of theum, material they were a little overloaded with just theamount of content that they— the information they weregiven in one session”. Faciliators felt that the literacylevel of the content and handouts, compared to the liter-acy levels of their participants was a barrier to providingsome of the recommended handouts; “I think for meum, just ethically when I'm presenting information I'malways conscious of literacy levels. So at some point Idid struggle with how we were presenting information,and just knowing some of the barriers that our familiesfaced, following through on homework and stuff likethat? I was a little hesitant with some of the material”. Inaddition, some cultural and social factors arose withinthe groups that affected how the facilitator implementedHT. These included; religious-based food restrictionsimpacting usability of recipes, the family traditions pro-gram content for children in care and concerns regard-ing the use of the word ‘family’ and the economicstruggles faced by some of the families and facilitatorsfeeling uncomfortable discussing purchasing certaintypes of food. A number of quotes highlight theseconcerns; "We have some families that, like, for like, reli-gious reasons they don't eat certain foods", and "Wewere very cognitive that we had children in care, andsome of their triggers. We cut out parts of the programbased on that". In addition some facilitators commented,"For me I struggled with the economics of, and the fi-nancial situations of many of our families. They mightwant to eat healthier but might not be able to afford thehealthy fresh vegetables, and fruits, and that kind ofthing. So not that they don't want to, but just can’t withwhat they’re given".Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 11 of 16Facilitators to implementationFacilitators felt the HT manual was easy to follow andcontained a lot of good information; "Very well orga-nized and prepared, easy to just take the book and beable to do the program with the family and the kids". Inaddition, many facilitators felt that they could adapt theprogram to their communities needs such as includinglocal fruits and vegetables to their region or adaptingrecipes to accommodate religious needs. Facilitators withmore experience reported a greater ease in modifyingthe content then those with less experience.Costs – TimeThe original manual suggested that facilitators allocated6 h per week to prepare for and deliver HT, whileprogram assistants should allocate 4 h. On average facili-tators spent 6.41 h per week prepping and deliveringHT, while program assistants spent 4.28 h per week onaverage prepping for HT. Facilitators reported that prep-aration took longer for HT as they had never deliveredthe program before and because they sought out alterna-tive resources to those provided within the manual.Costs - FinancialAs part of the project each site was allocated $36,000 toimplement the program. This money was provided tocover staff wages and program supplies, including phys-ical activity materials and cooking ingredients. Only twosites provided information regarding the cost of runningHT. One site reported that the program cost the full$36,000 to implement with the majority of money spenton staff wages ($27,412). Additional funds were spent onpurchasing resources ($3302) and miscellaneous costsincluding insurance, computers, promotion, rent andtransportation ($5958). The second site reported theprogram costing $14,059 to implement. Costs includedstaff wages ($12,362) and resources ($2141). Directorsnoted that HT cost a significant amount of money to im-plement given the cost of cooking ingredients and phys-ical activity resources. In addition, some sites had to rentspaces in which to conduct the cooking component ofthe program. The majority of sites felt that this programwas not sustainable without additional funding.MaintenanceSetting levelTwo implementation sites completed HT in full in theyear following initial program implementation (seeAdditional file 13 for maintenance data based on site).Five sites incorporated components of HT into existingprograms. One site had not implemented HT in the pre-vious year, or aspects of it, due to time constraints andleadership changes but planned to do so in the upcom-ing year with residual funds from the originalimplementation. Many directors and program coordina-tors felt that HT would not be feasible to run in its en-tirety without funding, with the majority of funds beingrequired for staff wages. In general directors felt that HTcould be incorporated into existing programs to reducecosts. One director felt that HT did not align with theneeds of their target population, specifically a trauma-exposed population, while all others felt that HT alignedwith their organizational mission.DiscussionThe current study utilized the RE-AIM framework toevaluate the internal and external validity of acommunity-developed and implemented healthy eatingand physical activity program designed for vulnerablechildren and their families. The evaluation identifiedprogram strengths as well as important areas for im-provement and demonstrates the challenges of conduct-ing community-based research.ReachHealthy Together (HT) reached a small portion ofeligible families. Although all sites stated serving vulner-able populations as part of their organization missiononly two sites expressed definitive eligibility criteriabeyond the HT age requirements. This was due to lowresponse rates from the target populations, leading tothe expansion of recruitment to all families within thesites catchment areas. Broadening the target populationlimits inferences regarding the reach and impact of HTon the vulnerable populations for which it wasdeveloped. While recruiting children for communityprograms is challenging [26] it is recommended thatprogram developers and implementers clearly define thetarget population before recruitment in order to estab-lish if the program is able to reach and positively impactthe specified population. In addition to recruitment chal-lenges, high attrition rates were reported, similar tothose seen in previous research with children of ethnicminorities and from low socio-economic families [27].The challenge of recruiting and retaining children fromvulnerable populations for community programs are notuncommon but are rarely reported by primary interven-tion studies [28, 29]. It is imperative that community sitesmake a concerted effort to employ directed strategies torecruit, engage and retain children and their families forsuch programs such as establishing trusting relationshipswith parents and children, utilizing a program championand offering participation incentives [30].EffectivenessHT increased caregiver’s confidence to engage in healthyeating practices and to cook. Furthermore, caregivers re-ported a reduction in the availability of unhealthy foodJung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 12 of 16available in the house immediately after HT and at6-month follow-up, replicating the finding of Robertsonand colleagues [31] in their 12-week family focused com-munity program. When examined by site, Sites A and Jappeared to demonstrate reductions in unhealthy foodavailability between pre- to post-program (Site A) andpre-program to 6-month follow-up.HT did not impact self or proxy reports of individualbehaviour, perceived social support for physical activityand healthy nutrition or the broader outcome of qualityof life. One explanation for the lack of changes to behav-iour, social support, and quality of life in general is theatheoretical nature of the program. A recent literaturereview demonstrated that theory-based interventions aremore successful than atheoretical approaches inchanging adolescent’s physical activity behaviour [32].Similarly, Hoelscher and colleagues [33] highlight theimportance of theory in designing effective nutrition in-terventions for adolescents. Interventions created with-out a guiding theoretical framework elude examinationand understanding of the causal mechanisms involved incomplex behaviour change such as nutrition and phys-ical activity [34]. Furthermore, without explicitly target-ing theoretical constructs of change establishing themost appropriate channels of program adaptation arechallenging [32]. An additional factor that may limit theeffectiveness of HT is the short-term nature of the pro-gram. Specifically, five 2-hour sessions may not havebeen sufficient to evoke long-term changes in the de-sired cognitive or behavioural outcomes. Similarcommunity-based family obesity prevention programsconducted over a greater time period have reportedpositive outcomes [15, 35]. It is recommended that fu-ture community-based behaviour change programs workclosely with behaviour change experts to co-develop the-oretically driven content and establish appropriate pro-gram duration to maximize the likelihood of promotingchange in outcomes of interest. While the overall impactof any health program cannot be predicted by effective-ness alone [36], it should be taken into considerationwhen contemplating the future of a program.ImplementationIn this iteration of HT (version 1), fidelity was moderatewith facilitators stating time as the biggest barrier to pro-gram implementation. Due to limited time, and the largeamount of content contained within HT, facilitators feltunable to implement all components. Observations fromsite A revealed that the percentage to which session objec-tives were met was low in a number of sessions due to thefacilitators dropping aspects of the program due to time,often the physical activity section. Interestingly, in site Acaregivers reported a decrease in children’s physical activ-ity behaviour from before to after HT as well as a decreasein the amount of social support they provided. The degreeto which nutrition-related components, such as cooking,were completed was 20% higher than physical activity re-lated components. Given that implementation fidelity caninfluence program effectiveness [37], higher fidelity couldexplain the more favorable outcomes reported for nutri-tion related outcomes in caregivers in comparison tophysical activity outcomes. Differences in implementationfidelity between nutrition and physical activity contentcould be due to the lack of experience of the facilitatorswith the program content. Specifically, the majority of fa-cilitators reported previous involvement in nutrition pro-grams; however, only 2 facilitators had previouslyfacilitated physical activity programs. It is possible that,given the time constraints, facilitators chose to drop thephysical activity portion of the program ahead of thecooking section due to comfort in facilitating the activity.The knowledge, experience and confidence of facilitatorsin delivering a program and its content are essential to thesuccess of a program [38, 39]. It is imperative that the ex-perience and confidence level of potential staff be exam-ined in order to plan appropriate training. Prior tofacilitating HT facilitators received 2 days of training thatfocused heavily on the process of group facilitation.Community-based programs targeting nutrition and phys-ical activity are encouraged to work with nutrition andphysical activity experts to provide facilitators with train-ing on program content to ensure facilitators feelknowledgeable and confident to implement the programas intended [15, 31, 35]. Given the extensive contentwithin HT, facilitators and directors believed that programeffectiveness and fidelity might benefit from extending thelength of the HT program, which the Bridge now raise asan option in their current HT facilitator trainings. Similarfamily community-based obesity prevention programshave ranged from 9-weeks [15] to a 4-week camp [35],with positive outcomes.MaintenanceDue to the perceived high cost of implementing HT manysites incorporated specific components of the programwithin other existing programs in the follow-up year. Com-munity sites are tightly bound by financial constraints anddetermining methods to reduce costs is essential if a pro-gram is to be sustained. Sites that worked in close collabor-ation with community partners, such as schools, were ableto reduce costs by utilizing school facilities. Furthermore,collaboration with schools assisted in the recruitment of anumber of participants. The development of communitypartnerships has been highlighted as essential in ensuringthe success of a program [40] and as such HT developersare providing future implementation sites with suggestionson how to engage potential community partners.Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 13 of 16Strengths and limitations of the study and communitybased program evaluationAs is common when conducting community-based in-terventions [40, 41], lack of reliable data and a controlgroup was a constraint in this evaluation and highlightsthe need for caution when reviewing findings. However,the use of multiple data sources to assess each RE-AIMcomponent in this study provided a substantial amount ofrich data, which serves to increase confidence in theconclusions made, and adds contextual relevance to thefindings. A second limitation is the inability to assessintervention dose response due to limited monitoring ofparticipant attendance at program sessions and the ‘drop-in’ format employed by some sites. A third limitation wasthe high rate of attrition, restricting full examination ofoutcome variables by site, as well as multi-level modelingto account for the nested populations. While effectivenesswas examined by site where possible the inability to fullyexamine effectiveness of HT at every site makes it ex-tremely difficult to identify links between site specific con-textual factors and program effectiveness. Data pertainingto the individual sites should be considered with cautiongiven the limited sample size and the potential for Type 1error. The lack of a control arm meant no causal infer-ences could be made. Ideally, more robust research, suchas a randomized controlled trial, is needed in order to de-termine the effectiveness of HT under ideal conditions.That being said, evaluation of real world community pro-grams enables the examination of external factors regu-larly neglected in rigorous intervention studies [42]. Thecurrent evaluation examines both external and internal in-dicators of program success in an attempt to provide abroad understanding of the impact of HT. It is clear that abalance needs to be established between the interests ofthe community and the desire for rigorous evaluationmethodology [43].The future of healthy togetherThis is the first evaluation of the first iteration of HT, afamily-education program designed and implemented bya community organization to address healthy weights invulnerable children. Given the growing evidence insupport of caregiver-inclusive interventions to addresspediatric obesity [44] HT has the potential to make apositive impact on its target audience if modications aremade. This evaluation provides the opportunity for HTdevelopers, and other community-based organizations,to address areas of concern in order to strengthen theoverall impact of the program before further dissemin-ation and implementation. Based on these findings theBridge have made a number of adaptations to HT,including a) providing facilitators with training on thespecific program content in order to increase knowledgeand confidence of program delivery and b) increasingthe length of the program from 5- to 10-weeks in orderfor all program content to be covered.ConclusionOverall Healthy Together represents a feasible community-based healthy weights initiative that can be success-fully implemented in a variety of populations acrossCanada. However, further refinement of the programis required in order to ensure the program is effectiveat positively impacting physical activity and healthyeating cognitions and behaviour. In is recommendedthat community organizations, such as the Bridge,work closely with behaviour change experts to ensurethat community designed interventions target key theoret-ical constructs found to promote behaviour change. Inaddition, program implementation and dissemination ex-perts should be consulted to increase potential of the pro-gram to positively impact public health. Simultaneouslythe continued evaluation from arms-length evaluators thatexamine changes in outcomes of interest through bothqualitative and quantitative means is warranted.Additional filesAdditional file 1: Overview of measures used in the Healthy Togetherevaluation, data collected and timeline. Description of what measures weretaken and when during the Health Together evaluation. (DOCX 59 kb)Additional file 2: Description of Child Survey. Explanation of the itemsused within the child and youth surveys and associated reliabilitystatistics. (DOCX 18 kb)Additional file 3: Description of Caregiver Survey. Explanation of theitems used within the caregivers surveys and associated reliabilitystatistics. (DOCX 19 kb)Additional file 4: Individual site analyses and details of which sites hadsufficient data to perform analyses (caregiver and child). (DOCX 25 kb)Additional file 5: Statistics related to the Reach of the Healthy Togetherprogram. Description of data used examine program reach. (DOCX 14 kb)Additional file 6: Demographic characteristics of the caregivers atbaseline based on site. (DOCX 21 kb)Additional file 7: Staff adoption of Healthy Together. Description ofinternal and external recruitment for Healthy Together across sites.(DOCX 54 kb)Additional file 8: Healthy Together staff demographics. (DOCX 47 kb)Additional file 9: Facilitator demographics by site. (DOCX 15 kb)Additional file 10: Evaluator’s Observations Module 1 (Data from 8implementation sites). Description of specific module and sessionoutcomes for Module 1 based on observations. (DOCX 16 kb)Additional file 11: Evaluator’s Observations Module 2 (Data from 10implementation sites). Description of specific module and sessionoutcomes for Module 2 based on observations. (DOCX 16 kb)Additional file 12: Evaluator’s Observations Module 3 (Data from 10implementation sites). Description of specific module and sessionoutcomes for Module 3 based on observations. (DOCX 16 kb)Additional file 13: Setting level maintenance of Healthy Together inthe past year. (DOCX 15 kb)AcknowledgementsThe authors would like to thank The Bridge Youth and Family Services forallowing us to evaluate their program.Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 14 of 16FundingThis program evaluation was funded by the Public Health Agency of Canada.Availability of data and materialsThe datasets used during the current study are available from thecorresponding author on reasonable request.Authors’ contributionsM.E.J. was responsible for project conception. M.E.J. and J.E.B. designed theevaluation, developed the overall evaluation plan, conducted the researchand analyzed the data. M.E.J., J.E.B and H.L.G determined the content for themanuscript and collaborated on the writing. All authors read and approvedthe final manuscript.Ethics approval and consent to participateThe University of British Columbia Research Ethics Board approved theprotocol. All participants provided consent to participate. Assent was alsoobtained for children under 16 years of age.Consent for publicationNot applicable.Competing interestsThe authors declare that they have no competing interests.Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.Author details1School of Health and Exercise Sciences, Faculty of Health and SocialDevelopment, University of British Columbia, Kelowna, Canada. 2Centre forExercise, Nutrition and health Sciences, School for Policy Studies, Universityof Bristol, Bristol, UK.Received: 30 May 2017 Accepted: 18 December 2017References1. Ball GD, McCargar LJ. Childhood obesity in Canada: a review of prevalenceestimates and risk factors for cardiovascular disease and type 2 diabetes.Can J Appl Physiol. 2003;28(1):117–40.2. Bridger T. Childhood obesity and cardiovascular disease. Paediatr ChildHealth. 2013;14(3):177–82.3. Freedman DS, Mei Z, Srinivasan SR, Berenson GS, Dietz WH. Cardiovascularrisk factors and excess adiposity among overweight children andadolescents: the Bogalusa heart study. J Pediatr. 2007;150(1):12–7.4. Weiss R, Dzuira J, Burgert TS, Tamborlane WV, Taksali SE, et al. Obesity andthe metabolic syndrome in children and adolescents. N Engl J Med.2004;350:2362–74.5. Singh AS, Mulder C, Twisk JW, van Mechelen W, Chinapaw MJ. Tracking ofchildhood over- weight into adulthood: a systematic review of theliterature. Obes Rev. 2008;9(5):474–88.6. Rodd C, Sharma AK. Recent trends in the prevalence of overweight andobesity among Canadian children. CMAJ. 2016;188(13):E313–20.7. Anis AH, Zhang W, Bansback N, Guh DP, Amarsi Z, Birmingham CL. Obesityand overweight in Canada: an updated cost-of-illness study. Obes Rev.2010;11(1):31–40.8. Public Health Agency of Canada. Obesity in Canada: health and economicimplications. 2011 www.phac-aspc.gc.ca/hp-ps/hl-mvs/oic-oac/econo-eng.php.Accessed 12 April 2017.9. Haas JS, Lee LB, Kaplan CP, Sonneborn D, Phillips KA, Liang SY. Theassociation of race, socioeconomic status, and health insurance status withthe prevalence of overweight among children and adolescents. Am J PublicHealth. 2003;93:2105–10.10. Bruner MW, Lawson J, Pickett W, Boyce W, Janssen I. Rural Canadianadolescents are more likely to be obese compared with urban adolescents.Int J Pediatri Obes. 2008;3(4):205–11.11. Bleich SN, Segal J, Wu Y, Wilson R, Wang Y. Systematic review ofcommunity-based childhood obesity prevention studies. Pediatr. 2013;https://doi.org/10.1542/peds.2013-0886.12. Glickman D, Parker L, Sim LJ, Del Valle Cook H, Miller EA. Acceleratingprogress in obesity prevention: solving the weight of the nation.Washington, DC: institute of. Medicine. 2012; https://doi.org/10.17226/13275.13. Oude Luttikhuis H, Baur L, Jansen H, Shrewsbury VA, O’Malley C, et al.Interventions for treating obesity in children. Cochrane Database Syst Rev.2009; https://doi.org/10.1002/14651858.CD001872.pub2.14. Spear BA, Barlow SE, Ervin C, Ludwig DS, Saelens BE, et al.Recommendations for treatment of child and adolescent overweight andobesity. Pediatr. 2007;120(4):S254–88.15. Sacher PM, Kolotourou M, Chadwick PM, Cole TJ, Lawson MS, et al.Randomized controlled trial of the MEND program: a family-basedcommunity intervention for childhood obesity. Obesity 2010; https://doi.org/10.1038/oby.2009.433.16. Foster GD, Sundal D, McDermott C, Jelalian E, Lent MR, Vojta D. Feasibilityand preliminary outcomes of a scalable community-based treatment ofchildhood obesity. Pediatr. 2012;130(4):652–9.17. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact ofhealth promotion interventions: the re-aim framework. Am J Public Health.1999;89:1322–7.18. Glasgow RE, Klesges LM, Dzewaltowski DA, Estabrooks PA, Vogt TM.Evaluating the impact of health promotion programs: using the RE-AIMframework to form summary measures for decision making involvingcomplex issues. Health Educ Res. 2006;21:688–94.19. Hingle MD, Turner T, Kutob R, Merchant N, Roe DJ, et al. The EPIC kidsstudy: a randomized family-focused YMVA-based intervention to preventtype 2 diabetes in at-risk youth. BMC Public Health. 2015; https://doi.org/10.1186/s12889-015-2595-3.20. Olstad DL, Ball K, Abbott G, McHaughton SA, Le HN, et al. A processevaluation of the supermarket healthy eating for life (SHELf) randomizedcontrolled trial. Int J Behav Nutr Phys Act. 2016;24:13–27.21. Spook JE, Paulussen T, Paulissen R, Visschedijk G, Kok G, van EmpelenP. Design rationale behind the serious self-regulation gameintervention “balance it”: overweight prevention among secondaryvocational education students in the Netherlands. Games Health J.2015;4(5):387–400.22. Greene JC, Benjamin L, Goodyear L. The merits of mixing methods inevaluation. Evaluation. 2001;7(1):25–44.23. Tabachnik BG, Fidell, LS. Using multivariate statistics. 6th ed. Boston: PearsonEducation Inc. 2013.24. Cohen, J. Statistical power analysis for the behavioral sciences, 2nd Edition.Hillside: Lawrence Erlbaum. 1988.25. Ritchie J, Lewis J. Qualitative research practice: a guide for social sciencestudents and researchers. Great Britain: Sage Publications Inc; 2003.26. Lamb J, Puskar KR, Tusaie-Mumford K. Adolescent research recruitmentissues and strategies: application in a rural school setting. J Pediatr Nurs.2001;16(1):43–52.27. Karlson CW, Rapoff MA. Attrition in randomized controlled trials for pediatricchronic conditions. J Pediatr Psychol. 2009;34(7):782–93.28. Backinger CL, Michaels CN, Jefferson AM, Fagan P, Hurd AL, Grana R. Factorsassociated with recruitment and retention of youth into smoking cessationintervention studies—a review of the literature. Health Educ Res.2008;23(2):359–68.29. Trapp G, Giles-Corti B, Martin K, Timperio A, Villanueva K. Conducting fieldresearch in a primary school setting: methodological considerations formaximising response rates, data quality and quantity. Health Educ J. 2011;11:1–7.30. Schoeppe S, Oliver M, Badland HM, Burke M, Duncan MJ. Recruitment andretention of children in behavioral health risk factor studies: REACHstrategies. Int J Behav Med. 2014;21:794–803.31. Robertson W, Friede T, Blissett J, Rudolf MCJ, Wallis M, Stewart-Brown S.Pilot of “families for health”: community-based family intervention forobesity. Arch Dis Child. 2008;93:921–6.32. McGoey T, Root Z, Bruner MW, Law B. Evaluation of physical activityinterventions in children via the reach, efficacy/effectiveness,adoption, implementation and maintenance (RE-AIM) framework: asystematic review of randomized and non-randomized trials. PrevMed. 2015;82:8–19.33. Hoelscher DM, Evans A, Parcel G, Kelder S. Designing effective nutritioninterventions for adolescents. J Am Diet Assoc. 2002;103(3):S52–63.34. Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reportingcomplex behaviour change interventions: the need for a scientific method.Implement Sci. 2009;4:1–6.Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 15 of 1635. Burke SM, Shapiro S, Petrella RJ, Irwin JD, Jackman M, et al. Using theRE-AIM framework to evaluate a community-based summer camp forchildren with obesity: a prospective feasibility study. BMC Obes. 2015;https://doi.org/10.1186/s40608-015-0050-8.36. Glasgow RE, Lichtenstein E, Marcus A. Why don't we see more translationof health promotion research to practice? Rethinking the efficacy toeffectiveness transition. Am J Public Health. 2003;93:1261–7.37. Durlak JA, DuPre EP. Implementation matters: a review of research on theinfluence of implementation on program outcomes and the factorsaffecting implementation. Am J Community Psychol. 2008;41:327–50.38. Glasgow RE, Klesges LM, Dzewaltowski DA, Bull SS, Estabrooks P. The futureof health behavior change research: what is needed to improve translationof research into health promotion practice? Ann Behav Med. 2004;27:3–12.39. Green LW, Glasgow RE. Evaluating the relevance, generalization, andapplicability of research: issues in external validation and translationmethodology. Eval Health Prof. 2006;29:126–53.40. Koorts H, Gillison F. Mixed method evaluation of a community-basedphysical activity program using the RE-AIM framework: practical applicationin a real-world setting. BMC Public Health. 2015;15:1102–12.41. Brownson RC, Brennan LK, Evenson KR, Leviton LC. Lessons from amixed-methods approach to evaluating active living by design. Am J PrevMed. 2012;43(5S4):S271–80.42. Klesges LM, Williams NA, Davis KS, Buscemi J, Kitzmann KM. External validityreporting in behavioral treatment of childhood obesity: a systematic review.Am J Prev Med. 2012;42:185–92.43. Cullen A, Giles T, Rosenthal J. Evaluating community-based child healthprograms: a snapshot of strategies and methods. Portland, ME: NationalAcademy for State Health Policy. 2006.44. Faith MS, Van Horn L, Appel IJ, Burke LE, Carson JA, et al. Evaluating parentsand adult caregivers as “agents of change” for treating obese children:evidence for parent behavior change strategies and research gaps: ascientific statement from the American Heart Association. Circulation.2012;125(9):1186–207.•  We accept pre-submission inquiries •  Our selector tool helps you to find the most relevant journal•  We provide round the clock customer support •  Convenient online submission•  Thorough peer review•  Inclusion in PubMed and all major indexing services •  Maximum visibility for your researchSubmit your manuscript atwww.biomedcentral.com/submitSubmit your next manuscript to BioMed Central and we will help you at every step:Jung et al. International Journal of Behavioral Nutrition and Physical Activity  (2018) 15:13 Page 16 of 16


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items