UBC Faculty Research and Publications

A formative evaluation of the implementation of an upper limb stroke rehabilitation intervention in clinical… Connell, Louise A; McMahon, Naoimh E; Harris, Jocelyn E; Watkins, Caroline L; Eng, Janice J Aug 12, 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-13012_2014_Article_90.pdf [ 262.58kB ]
JSON: 52383-1.0223775.json
JSON-LD: 52383-1.0223775-ld.json
RDF/XML (Pretty): 52383-1.0223775-rdf.xml
RDF/JSON: 52383-1.0223775-rdf.json
Turtle: 52383-1.0223775-turtle.txt
N-Triples: 52383-1.0223775-rdf-ntriples.txt
Original Record: 52383-1.0223775-source.json
Full Text

Full Text

RESEARCHA formative evaluation ofaliis2Background not achieving the required intensity to maximise recoveryImplementationScienceConnell et al. Implementation Science 2014, 9:90http://www.implementationscience.com/content/9/1/90family/carer where possible. The program is not meantLancashire, Preston PR1 2HE, UKFull list of author information is available at the end of the articleIn recent years, there has been a dramatic increase instroke rehabilitation research and the evidence-basehas grown exponentially. High-intensity, repetitive, task-oriented training demonstrates the best evidence forimproving motor recovery after stroke [1-4]. However, itis known that stroke rehabilitation, in its present form, isafter stroke [5,6].The Graded Repetitive Arm Supplementary Program(GRASP) is one method of increasing intensity of exerciseduring inpatient rehabilitation. In 2009, GRASP was shownto be significantly more effective in promoting functionalrecovery of the upper limb after stroke compared to usualcare [7]. GRASP is a self-directed hand and arm exerciseprogramme that is taught and monitored by a therapist,but carried out by the patient with the support of their* Correspondence: laconnell@uclan.ac.uk1Clinical Practice Research Unit, School of Health, University of CentralAbstractBackground: The Graded Repetitive Arm Supplementary Program (GRASP) is a hand and arm exercise programmedesigned to increase the intensity of exercise achieved in inpatient stroke rehabilitation. GRASP was shown to beeffective in a randomised controlled trial in 2009 and has since experienced unusually rapid uptake into clinicalpractice. The aim of this study was to conduct a formative evaluation of the implementation of GRASP to informthe development and implementation of a similar intervention in the United Kingdom.Methods: Semi-structured interviews were conducted with therapists who were involved in implementing GRASPat their work site, or who had experience of using GRASP. Normalisation Process Theory (NPT), a sociological theoryused to explore the processes of embedding innovations in practice, was used to develop an interview guide.Intervention components outlined within the GRASP Guideline Manual were used to develop prompts to explorehow therapists use GRASP in practice. Interview transcripts were analysed using a coding frame based onimplementation theory.Results: Twenty interviews were conducted across eight sites in British Columbia Canada. Therapists identifiedinformal networks and the free online availability of GRASP as key factors in finding out about the intervention. Alltherapists reported positive opinions about the value of GRASP. At all sites, therapists identified individuals whoadvocated for the use of GRASP, and in six of the eight sites this was the practice leader or senior therapist. Rehabilitationassistants were identified as instrumental in delivering GRASP in almost all sites as they were responsible for organisingthe GRASP equipment and assisting patients using GRASP. Almost all intervention components were found to beadapted to some degree when used in clinical practice; coverage was wider, the content adapted, and the dose, whenmonitored, was less.Conclusions: Although GRASP has translated into clinical practice, it is not always used in the way in which it wasshown to be effective. This formative evaluation has informed the development of a novel intervention which aims tobridge this evidence-practice gap in upper limb rehabilitation after stroke.an upper limb stroke rehin clinical practice: a quaLouise A Connell1*, Naoimh E McMahon1, Jocelyn E Harr© 2014 Connell et al.; licensee BioMed CentraCommons Attribution License (http://creativecreproduction in any medium, provided the orDedication waiver (http://creativecommons.orunless otherwise stated.Open Accessthe implementation ofbilitation interventiontative interview study, Caroline L Watkins1 and Janice J Eng3l Ltd. This is an Open Access article distributed under the terms of the Creativeommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andiginal work is properly credited. The Creative Commons Public Domaing/publicdomain/zero/1.0/) applies to the data made available in this article,Connell et al. Implementation Science 2014, 9:90 Page 2 of 12http://www.implementationscience.com/content/9/1/90to replace existing therapy services, rather to augmentcurrent therapy, adding opportunities for more practice.Despite only one randomised controlled trial (RCT) hav-ing demonstrated the efficacy of GRASP, a recommenda-tion to ‘provide a graded repetitive arm supplementaryprogram for patients to increase activity on ward and athome’ was included in the 2010 update of the CanadianBest Practice Recommendations for Stroke Rehabilitation[8]. Anecdotally GRASP is reported to be used in over 30centres in Canada [9] and from a sample of 274 therapistsin the United Kingdom (UK), over 40% had heard ofGRASP and almost one-quarter had experience of usingGRASP in practice [10].As the long-term objective of this work is to develop afeasible and structured upper limb exercise programmefor use in UK stroke rehabilitation units, it is of value tolearn from the implementation of GRASP. Stetler et al.[11] have highlighted the role of formative evaluation inimplementation research, defining it as ‘a rigorousassessment process designed to identify potential andactual influences on the progress and effectiveness ofimplementation efforts.’ There is growing interest inbeing able to systematically explore and explain theimplementation of evidence-based interventions and thishas resulted in the development a number of theoreticalframeworks. It has been suggested that the use of suchframeworks will help advance implementation research byproviding consistency in definitions and terminologiesacross contexts [12], and by providing systematic ex-planations of phenomena and constructs that influenceimplementation [13]. There is also increasing emphasisbeing placed on evaluating implementation fidelity[14], as each time implementation of an intervention isattempted, there is an opportunity to learn about condi-tions that result in better or worse fidelity, in order toassist refinement [15].The aim of this study was to conduct a formativeevaluation of the implementation of the Graded RepetitiveArm Supplementary Program (GRASP) in Vancouver,British Columbia (BC), Canada to inform the developmentand implementation of a similar intervention in the UK.The objectives of this study were to use semi-structuredinterviews to:1. Explore how therapists found out about GRASP.2. Explore the processes that therapists’ reportwere involved in implementing GRASP inpractice.3. Explore therapists’ experiences of using GRASP inclinical practice and how this adheres to interventioncomponents outlined within the GRASP GuidelineManual.4. Use a taxonomy of factors influencing implementationto explain the research findings.MethodsResearch team and reflexivityThe first author (LAC) and second author (NEM) con-ducted the interviews. Both are female-chartered phys-iotherapists with previous experience of qualitative datacollection. Both hold full-time research positions at aUK Higher Education Institution working on a NationalInstitute for Health Research (NIHR) funded project todevelop a clinically feasible structured upper limb exerciseprogramme for use in National Health Service (NHS)stroke rehabilitation units. The researchers were notknown to the participants prior to the study. Participantswere informed in the first recruitment email that two re-searchers from the UK were exploring how GRASP hasbeen implemented in practice. The third (JH) and last (JE)author developed GRASP and conducted the randomisedtrial confirming its effectiveness [7]. The fourth (CW) au-thor is a health services researcher with experience in im-plementation science.Study designA cross-sectional study design was used with data collectedvia semi-structured interviews.Theoretical frameworkThe approach used in this study was directed contentanalysis, a qualitative approach that is guided by a struc-tured process underpinned by theory [16]. Three frame-works from implementation science were used to addressthe study objectives and are detailed below.Normalisation Process Theory (NPT)Normalisation Process Theory (NPT) is a sociologicaltheory that can be used to understand the implementation,embedding, and integration of innovation in healthcaresettings [13]. NPT is made up of four constructs each ofwhich has four components: coherence is the first con-struct, and describes the sense-making processes thatpeople go through when introduced to a new innovation;cognitive participation describes the process of commit-ting to implementing the innovation; collective actiondescribes how the work to implement the intervention getsdone; and reflexive monitoring describes the evaluationwork that takes place. The emphasis of these componentsis on the dynamic and interactive processes that take placewhen attempting to embed a new innovation or practice. Arecent systematic review found that in most cases NPT hasbeen used as an organising framework for analyses andreporting of findings in health research [17]. It has alsobeen used to inform study/intervention design, to generateresearch questions for fieldwork, and to create tools forinvestigating and supporting implementation [17]. Inthis study, NPT was used in developing the interviewguide and in data analysis to explore the processesConnell et al. Implementation Science 2014, 9:90 Page 3 of 12http://www.implementationscience.com/content/9/1/90involved in identifying, integrating, and embedding GRASPin practice.Conceptual Framework for Implementation Fidelity (CFIF)Carroll et al. developed the Conceptual Framework forImplementation Fidelity (CFIF) to guide the measurementof implementation fidelity [18]. Within this framework,the elements of implementation fidelity are: coverage(who should be receiving the intervention); content (theintervention itself); and dose (duration and frequency ofthe intervention). The degree to which these elements aredelivered can be influenced by moderating factors, e.g.,intervention complexity, facilitation strategies, and partici-pant responsiveness. The CFIF has previously been usedempirically to evaluate fidelity [14,19]. In this study, theCFIF was used to analyse interview transcripts to explorethe coverage, content, and dose when GRASP is usedin clinical practice, and how this adheres to interven-tion components outlined within the GRASP GuidelineManual.Consolidated Framework for Implementation Research (CFIR)The Consolidated Framework for Implementation Research(CFIR) has been developed by Damschroder et al. and is apragmatic taxonomy of the factors that influence imple-mentation [12]. CFIR has five domains (characteristics ofthe intervention, inner setting, outer setting, characteristicsof individuals, and processes), each of which contain anumber of constructs. The framework can be used to guideassessments of implementation, evaluate implementationprogress, and explain findings in research studies [12]. Inthis study, CFIR was used in data analysis to identifyemerging factors that influenced implementation and useof GRASP, and to propose potential explanations for theresearch findings.Participant selectionA purposive sample of physical therapists, occupationaltherapists and rehabilitation assistants: who were cur-rently using GRASP, or had previous experience of usingGRASP, or who were involved in the implementation ofGRASP at their work setting were recruited to take partin this study. Therapists and work settings that were notusing GRASP or did not have experience of implement-ing GRASP in practice were not eligible for inclusion.Potential participants were identified through existingcontacts with the research team (e.g., through sites in-volved in the GRASP RCT), through the public registriesfor BC Occupational Therapists and Physical Therapistsand through a database of therapists that had agreed to becontacted about future research relating to the programthrough the GRASP website. These potential participantswere sent an email by the fifth author (JE) outlining thedetails of the study and inviting them to take part in aninterview of maximum one hour in length. A snowballsampling technique was used in which these participantsidentified colleagues from their own work place, or fromother sites in the region, who would be suitable to takepart.SettingThe interviews were conducted by the first (LAC) andsecond author (NEM). Interviews took place at the worksite of participants at a time deemed suitable by them.In instances where it was not possible to conduct the in-terviews face-to-face, the interviews were carried outover the telephone.Data collectionThe data collection tool used in this study was an inter-view guide (see Additional file 1: Interview Guide). NPTwas used to devise questions and prompts about theprocesses of implementing and embedding GRASP inpractice. Following introductory questions, participantswere asked how they found out about GRASP and todescribe in their own words how they use GRASP in prac-tice. The GRASP Guideline Manual (http://neurorehab.med.ubc.ca/grasp/) was used to identify components ofGRASP against which fidelity could be evaluated, andthese were included as prompts within the interviewguide. For example, in the GRASP Guideline Manual pg.8line 9 the instruction given is ‘Show patient and familyhow to do each exercise.’ A prompt relating to the familyand carer involvement component of GRASP was devel-oped for inclusion in the interview guide. The interviewguide was reviewed and piloted with researchers (n = 2)with previous experience of using implementation frame-works for semi-structured interviews, and with therapists(n = 3). The interviews lasted a maximum of one hour.They were audio-recorded and field notes made. All par-ticipants provided written informed consent and receiveda $25 Canadian honorarium to compensate them for theirtime. Interviews were conducted until no new implemen-tation issues were being reported and data saturation wasdeemed to have been reached.Data analysisInterview transcripts were transcribed verbatim andimported into NVivo 10 for analysis. Transcripts werefirst read for understanding to describe each case andto establish an initial coding frame. The coding framewas also informed by prior research that exploredupper limb exercise prescription by UK therapists, anduptake of GRASP in the UK, as it was hypothesisedthat similar experiences would arise for both populationgroups [10,20]. Transcripts were then re-read by the firstand second authors and separately coded.Connell et al. Implementation Science 2014, 9:90 Page 4 of 12http://www.implementationscience.com/content/9/1/90The coding frame evolved as analysis progressed (seeAdditional file 2: Coding Frame). This was facilitated byregular team meetings to discuss and agree on emergingthemes and resolve discrepancies in coding. NPTconstructswere used to code text relating to the processes of imple-menting GRASP in clinical practice. The CFIF was used tocode text relating to how GRASP is used in practice. Thesecodes were then used to evaluate adherence to the inter-vention components identified a priori from the GRASPGuideline Manual. The CFIR was used to code emergingfactors that influenced both use and implementation ofGRASP. Therapists in the research team provided feedbackthroughout the process, which helped to ensure thatfindings were credible.Ethical approvalThis study was approved by the University of BritishColumbia Behavioural Research Ethics Board (BREB),study number H13-00249.FindingsIn total 42 potential participants across 12 sites were in-vited to take part. Of these, 23 replied to the email invite,and 20 therapists from eight different sites agreed to takepart in an interview (two were not using GRASP and onereplied after data collection had ceased). Non-participantsdid not reply to the email invite. The reasons for non-participation are therefore unknown and it is not possibleto determine whether or not non-responders were imple-menting GRASP. Participant characteristics are shown inTable 1 along with their anonymised identification codes.For details on individual recruitment across sites seeAdditional file 3: Recruitment of participants. Therapistsfrom eight of the twelve contacted sites participated.Seven of these sites were regional hospitals and one sitewas a rehabilitation centre. Two of the eight sites weresituated in the Greater Vancouver area.How therapists found out about GRASPThe way in which each therapist found out about GRASPis shown in Table 1. Two therapists had acted as site co-ordinators for the RCT; 11 therapists found out aboutGRASP through colleagues or work in-services; one ther-apist learned about GRASP at a national physiotherapyforum; two therapists learned about GRASP as studentsin university; two therapists found out about GRASPthrough their own research; and two therapists found outabout GRASP through the research team at GF Strong.Processes involved in implementing GRASP in practiceCoherenceFirst impressions of GRASP were predominantly positivewith almost all therapists expressing that they felt GRASPwas well supported by the evidence base, was wellhad two at the time, practice leaders who oversee allof OTs in the building and they really kind of initiatedit…’ #OT4‘…it helped to implement things pretty quicklybecause our professional practice leader, who overseesall the OT’s in a bunch of our sister Hospitals, shewas very helpful and supportive…’ #OT13As therapists had concerns about quality of unsupervised#PT2‘…I would hesitate to even give it to someone athome. I mean, it’s meant to be a home program,but if all you’re doing is reinforcing that…that touse that tone to do it…I don’t think it’s benefitingthem and so I would…I would hesitate to sendsomeone home that’s not able to do it correctly.’#OT10Cognitive participation & collective actionTherapists identified key individuals at each site who initi-ated and/or supported the implementation of GRASP. Insix of the eight sites this individual was a clinical supervisoror practice leader:‘I think the practice leaders did, so in OT practice wedid have significant results so I think I was just amfelt happy about the research findings that they hadfrom the study.’ #OT2‘I thought it was great, I like that there was bigpictures, the writing is big, it’s very well laid out, veryeasy to give out as a home programme once youintroduce somebody to it and then to give to them todo on their own.’ #PT3However, all therapists interviewed also expressed someconcerns about the quality of exercises that patients wouldbe able to complete outside of therapy time:‘I thought it was a good idea that they weregetting extra practice, one of my initial concernswas the quality of the movement because we arealways so concerned that we want to get themto move as biomechanically proper as possible…’presented and that it was something that would helpthem in their role:‘I remember being impressed just by the researchfindings, so I remember that was highlighted that itdid show the extra practice on top of therapy sessionsexercises, rehabilitation assistants were almost alwaysinvolved in delivering GRASP to patients:duBCDEFGHConnell et al. Implementation Science 2014, 9:90 Page 5 of 12http://www.implementationscience.com/content/9/1/90Occupational therapist (#OT1) 9 BScOccupational therapist (#OT2) 3 MScOT practice leader (#OT3) 30 BScTable 1 Participant characteristicsSite Professional title Years of experience Level of eA‘I don’t know if anyone has ever done them correctlythe first time through, so I generally try to review itwith them a couple of times if I can myself, I alwaysdo it once…I would get a rehab assistant to go over itwith them a few times afterwards until they reallycould do them without sort of assistance.’ #OT5‘… if they’re not doing the full booklet then she [theOT] will tell me which exercises she wants me to dowith the patient and am from there if through workingwith them if I find that they’ve progressed or regressedOccupational therapist (#OT4) 25 BScPT practice leader (#PT1) 36 BScPT practice leader (#PT2) 11 BScOccupational therapist (#OT5) 6 BScOT practice leader (#OT6) 22 MScPhysiotherapist (#PT3) 5 BScOccupational therapist (#OT7) 12 BScRehabilitation assistant (#RA1) 6 CertOccupational therapist (#OT8) 19 BScRehabilitation assistant (#RA2) 8 CertOccupational therapist (#OT9) 8 MScOccupational therapist (#OT10) >5 BScPhysiotherapist (#PT4) 3 MScPhysiotherapist (#PT5) 4 MScOccupational therapist (#OT11) >5 BScOT practice leader (#OT12) 37 BScOccupational therapist (#OT13) 15 BSccation Introduction to GRASP Experience with GRASPRCT Involved in RCT, currently usingGRASP in community settingUniversity Involved in implementing GRASPin acute care setting, currentlyusing GRASP in inpatientrehabilitationColleagues/work in-service Involved in implementing GRASPat site AI can, I’ll then let the OT know and we can either addor take away.’ #RA1Acquiring the necessary equipment was identifiedas the most challenging process in implementingGRASP:‘That was probably actually the biggest barrier andthat’s probably why most therapists didn’t do it beforebecause it is a lot of little tiny things that you need tocollect…’ #OT5Colleagues Previous experience of usingGRASP in inpatient rehabilitation,not using GRASP in current roleResearch Team at GF Strong Involved in implementing GRASPat site APhysiotherapy Forum Involved in implementing GRASPat site B, currently using GRASP ingroups in inpatient rehabilitationColleagues/work in-service Has experience of using GRASPin inpatient rehabilitationColleagues/work in-service Involved in implementing GRASPat site CColleagues/work in-service Has experience of using GRASPin acute, inpatient rehabilitationand outpatient settingsColleagues/work in-service Using GRASP in acute care andinpatient rehabilitationColleagues/work in-service Using GRASP in inpatientrehabilitation, has experienceof using GRASP in outpatientsColleagues/work in-service Using GRASP in inpatientrehabilitationColleagues/work in-service Using GRASP in inpatientrehabilitationResearch Team at GF Strong Using GRASP in outpatientsColleagues/work in-service Using GRASP in outpatientsUniversity Using GRASP in acute careColleagues/work in-service Using GRASP in acute careOwn research Using GRASP in communitysettingOwn research Involved in implementing GRASPat site GRCT Involved in RCT and inimplementing GRASP at site HConnell et al. Implementation Science 2014, 9:90 Page 6 of 12http://www.implementationscience.com/content/9/1/90seems to work for us and our patients…’ #OT8Factors influencing the implementation and use of GRASPThe Consolidated Framework for Implementation Research(CFIR) was used to identify the most influential factors forusing, and implementing, GRASP in practice. These factorsare summarised in Table 3.DiscussionThe aim of this study was to conduct a formative evalu-ation of the implementation of the Graded Repetitive ArmSupplementary Program to inform the development of astructured upper limb exercise program in the UK. Theput them together. They chose items that werecommonly used, but hard to get…’#OT1How the GRASP was used in practiceTherapists, when implementing GRASP in practice, havemodified the intervention to fit with their clinical reason-ing and the environment in which they work. A summaryof how therapists’ use of GRASP differs from the interven-tion components identified within the GRASP GuidelineManual is shown in Table 2.Reflexive monitoringAppraisal processes most often occurred at the level ofthe individual. Therapists were often only able to describetheir own use and experiences with GRASP as opposed tomore collective appraisal at a team or department level:‘I’m not sure actually, I think a lot of people they justdo it as it’s laid out, I’m not sure if there is the samelevel of customisation, am…but I can’t say for sure.’#OT4‘I think we all do our own thing…I don’t know if weall do it…I think we all sort of just tweaked it to what‘We had one set of things to use as demonstration inour room in our gym but for the families they wouldhave to go and individually buy all the stuffthemselves.’ #PT2In the four sites that were able to provide GRASPequipment, it was the rehabilitation assistants who wereresponsible for the process:‘I actually stock pile all the equipment and we havelittle bags, like little back packs and we fill the rightequipment for the right patient at that time…’ #RA2‘So, they put together what are called starter kits at(one of the sites) and the RA’s, the rehab assistants,free online availability of the treatment protocol, alongwith well-established networks between the research andclinical teams, enabled therapists to find out about GRASP.All therapists expressed having positive first impressions ofGRASP, but also reported that they had some concernsabout prescribing exercises to be completed outside of ther-apy time. At each site, key individuals were identified whowere responsible for driving the implementation of GRASP,and in the majority of sites this individual was the practiceleader or clinical supervisor. All components of the GRASPintervention were modified to some extent when im-plemented in practice. Coverage was wider, the contentadapted and the dose, when monitored, was less. Ther-apists, although providing comprehensive appraisal ofthe implementation and use of GRASP from their ownperspective, were often unable to detail how GRASPwas being used at a team or departmental level. Factorsthat emerged as influential for the implementation anduse of GRASP have been identified.The free online availability of the GRASP materialsemerged as an important factor for therapists in findingout about the intervention. McCluskey et al. have identi-fied that the paucity of detailed information on how toimplement effective interventions acts as a barrier toimplementing stroke guideline recommendations [21].Within this article, it is suggested that researchers berequired to make the protocols of effective interventionsreadily available to practitioners to overcome this barrier.Therapists most often found out about GRASP throughexisting internal and external networks with colleaguesand the research team at GF Strong. Interestingly, themost frequent method of finding out about GRASP in theUK was also through colleagues [10]. Use of diverse for-mal and informal routes to acquire research knowledgehas been previously reported [22] and is reflective of the‘mindlines’ concept, in which healthcare professionals’decision making is most often informed by ‘by their inter-actions with each other and with opinion leaders’ [23].Therapists in this study identified key individuals ateach site who took responsibility for driving the imple-mentation of GRASP. In the majority of cases, these indi-viduals were more senior therapists and practice leaders.A recent realist review on this topic investigated thecomplex interactions between change agents, knowledgeutilisation, and work settings [24]. It was concluded thatalthough evidence for the effectiveness of change agentswas found to be weak, there was evidence to support theimportance of these roles. However, a lack of systematicreporting of change agency interventions limited the con-clusions that could be drawn, particularly in relation topersonal characteristics of change agents and the extent towhich they can be modified. More recently, Farley et al.have highlighted the challenge of collecting sufficiently de-tailed data to reliably and objectively identify high-qualityopinion leaders within the health services [25]. It wasour experience when travelling to the individual sitesConnell et al. Implementation Science 2014, 9:90 Page 7 of 12http://www.implementationscience.com/content/9/1/90Table 2 Therapists’ use of GRASP in clinical practiceto conduct the interviews in this study that althoughthe individuals responsible for introducing GRASP at eachsite were extremely enthusiastic about its implementationCoverage (who should receive the intervention)Intervention component from GRASP Guideline Manual1 Provide GRASP to stroke survivors in rehabilitation who can activelyelevate their scapula against gravity and have palpatable wristextension (grade 1); are aware of their safe bounds of ability;have sufficient cognition to be able to follow the programme;are able to report pain or fatigueContent (the content of the intervention)Intervention component from GRASP Guideline Manual2 Provide a GRASP manual which includes unilateral and bilateralstrengthening, range of motion, weight-bearing, and trunk controlexercises along with gross and fine motor exercises3 Provide a variety of GRASP equipment which can be substituted4 Provide a log sheet to monitor time spent completing exercises5 Progress to next GRASP level when the patient can complete over50% of the exercises in the current level6 Advise to complete the GRASP exercises outside of therapy time7 Encourage to keep moving their paretic arm as best they can,improper movement should not be the cause of omitting anexercise8 Teach GRASP exercises to family/carers were possibleDose (frequency and duration)Intervention component from GRASP Guideline Manual9 Advise to do the GRASP exercises for 60 minutes five times per weekand use, it was evident from informal conversations thatnot all therapists at the site were as enthused by theprogram. This would lead us to believe that althoughTherapists useGRASP was reported to be used not only in stroke rehabilitationunits but it is also used in acute care (n = 2), outpatient (n = 2),and community settings (n = 2); and with other populationgroups with neurological conditions.One therapist reported using the Fugl-Myer to select theappropriate GRASP level for each patient; the remainder selectedthe appropriate level based on observation of active movementand tone.Therapists useOne therapist reported always providing the full GRASP manualto patients. The majority of therapists selected the mostappropriate exercises from the manuals and printed them offindividually.Two sites provide full kits of equipment, one site provides halfsets of equipment which are the more difficult pieces to source(e.g., donut weight for hand), one provides equipment piece bypiece as needed, two use gym equipment that is cleaned andreused and two sell pieces of equipment to patients(e.g., theraputty).Six therapists mentioned using/trying to use a written checklistor log sheet to monitor exercise completed. The remainder usedverbal feedback from the stroke survivor and the clinical team tomonitor whether or not exercises were being completed.As therapists do not always use the full GRASP manual,progression was discussed in terms of adding in new sheets ofexercises or increasing repetitions as opposed to more structuredprogression through the levels of manuals.Nine therapists reported that stroke survivors, where able, wouldbe advised to complete exercises outside of therapy time. Barriersto prescribing exercises to be completed outside of therapy timeincluded therapists’ beliefs about patients’ ability to correctlycomplete exercises, patient safety awareness, cognitive impairmentand lack of family support for self-directed exercise. As a resultGRASP exercises were most often completed with the supervision/assistance of a rehabilitation assistant.All therapists made references to concerns they had about thequality of the exercises that stroke survivors would do and theamount of compensation. Exercises are regularly modified oromitted if it was felt that they were not being done correctly—particularly exercises resulting in shoulder hiking.All therapists reported that family played an important role inGRASP. The readiness and willingness of family members, asdetermined by the therapists, would influence the extent towhich they would be involved. A systematic approach to involvingfamily members or carers in rehabilitation was not reported.Therapists usePatients were advised by therapists to carry out the exercises asmuch as they could tolerate on a daily basis, rather than specifying60 minutes daily. Therapists discussed different approaches to gettingpatients to complete the desired amount of practice, such as splittingGRASP up throughout the day and providing extra sessions with therehabilitation assistant.Table 3 Factors influencing the implementation and useof GRASPInner and outer settingAccess to knowledge andinformationTen therapists reported that theGRASP website and free onlineavailability of the treatmentprotocol enabled them tofind out about the interventionand also facilitated its continueduse.Cosmopolitanism Therapists reported finding outabout GRASP through existingnetworks with the research teamat GF Strong (where GRASP wasdeveloped) and national meetingswith 11 therapists mentioningJanice Eng by name.Leadership engagement The implementation of GRASP wasfacilitated by active engagementof practice leaders and clinicalsupervisors as they were responsibleboth for identifying the programmeand introducing it at the work siteby acquiring resources to supportimplementation e.g., funding forequipment.Intervention characteristicsDesign, quality and packaging GRASP was perceived to be welldesigned and presented. The largetext and clear pictures were seento be highly beneficial, particularlyfor a population often suffering fromsome degree of cognitive impairment.Therapists reported that the manualcould be improved by shortening itand reducing repetition of exerciseswithin and between levels of manuals.Evidence strength and quality All therapists agreed that GRASPwas underpinned by best evidencefor motor recovery after stroke andreported sharing this informationwith the patients to whom theyprescribed GRASP.Relative advantage The primary advantage of GRASPwas that it provided a more timeefficient way of providing exercisesto patients – something that therapistsregularly do in practice anyway.Complexity Organising the GRASP equipmentwas identified as the most complexcomponent of the intervention andthis influenced the way in which theintervention was used i.e., substitutingitems of equipment or omitting someexercises altogether.Characteristics of individualsKnowledge and beliefs Therapists’ beliefs about the qualityof exercises that patients would beable to complete outside of therapytime influenced the way in whichGRASP was used in practice e.g.,completing GRASP exercises duringtherapy time.Connell et al. Implementation Science 2014, 9:90 Page 8 of 12http://www.implementationscience.com/content/9/1/90the uptake and implementation of GRASP appears highin the province, the number of therapists within individualdepartments actually consistently using the interventionmay not be as high, but to confirm this objectively a differ-ent study design is needed.It is of particular interest to note in this study, thatalthough the implementation of GRASP was found to begenerally good, i.e., all sites interviewed had successfullyintroduced GRASP to some extent into routine clinicalpractice, fidelity to the components outlined in GRASPGuideline Manual was lower than expected. It was foundthat all components of GRASP, when implemented inpractice, were adapted to some extent to fit with therapists’concerns about self-directed exercise and their workingcontext. The multi-faceted nature of GRASP, and the de-sign of the RCT in which it was tested, has meant that ithas not yet been possible to determine which component(s), i.e., those listed in Table 2, were the ‘active ingredient(s)’and contributed to the overall success of the programme[7]. Harn et al. [26] have recently discussed this topic withrespect to educational research and outline that interven-tions designed as a package that have been empiricallytested become evidence-based practice when it is stillunknown which components of the package are criticalfor success. Different schools of thought exist on adaptingevidence-based interventions, but it is now known thatadaptability of an intervention improves uptake and im-plementation [12,26,27].In this study, therapists’ beliefs about self-directedpractice emerged as one of the most influential factorsfor adapting GRASP when used in clinical practice. Des-pite the fact that the GRASP trial evidence showed thatpatients improved their function and movement quality[7], over one-half of the therapists expressed concernsthat exercise completed without therapist supervisionmight result in poor-quality movement. These concernsstem in part due to long-standing, but unfounded, beliefsthat practice of abnormal movement patterns promotepoor movement quality [28]. This finding has parallelswith fidelity studies from educational research whereteachers’ individual teaching philosophy and concernsabout interventions were found to moderate fidelity i.e.,teachers with more concerns about the value of the inter-vention demonstrated lower levels of implementation fi-delity [29,30]. Divergent views of how an intervention fitswith the role of those responsible for its implementationhave also been identified as a significant barrier to imple-mentation in two recent process evaluation studies usingversions of NPT [31,32]. It is becoming increasingly evidentthat the congruence or ‘fit’ of an intervention with thebeliefs of those delivering the intervention will determinesuccess. Although non-adopters were not interviewed inthis study, one could hypothesise that this perceived lackof congruence could, in part, explain non-adoption ofConnell et al. Implementation Science 2014, 9:90 Page 9 of 12http://www.implementationscience.com/content/9/1/90GRASP in other sites. Is there a case to be made foraddressing provider beliefs/concerns about evidence basedinterventions, or as Harn et al. suggest, is it time that em-pirically tested evidence-based interventions are adaptedto better match individual contexts to optimise implemen-tation fidelity [26]?In recent years there has been a substantial increase inresearch studies seeking to influence professional practice.A number of Cochrane reviews exist that have aimed toestablish the value of interventions such as computergenerated reminders [33], printed educational materials[34], audit and feedback [35], and continuing educationmeetings and workshops [36]. An important finding inthis study, when attempting to identify strategies to influ-ence practice, was the level at which therapists appraisedthe implementation and use of GRASP. It was found thattherapists were often unable to identify who in the depart-ment was using GRASP, and the way in which they wereusing the programme. Arguably it is difficult to influenceservice delivery, and therefore improve implementationfidelity, when service providers are not aware of whatcurrent practice is and do not benchmark or measure per-formance. This finding would suggest that there may be arole for self-monitoring, in the form of audit and feedbackfor example, to establish current practice and thus promptfidelity to treatment guidelines. Audit and feedback hasbeen shown to result in small but potentially importantimprovements [35] and recommendations as to how fu-ture empirical studies can further our understanding ofthe mechanisms of action of this complex interventionhave been proposed [37].Use of implementation frameworksAs no one implementation framework was identified thatcould address all of the study objectives, three separateimplementation frameworks were used to explore the pro-cesses of implementation, how GRASP is used in practiceand emerging factors which influenced implementationand use. Using three implementation frameworks, althoughcomplicated, provided a systematic way of capturing thecomplex aspects of implementation.NPT was useful in developing an interview guide toexplore the implementation process at each site. However,using NPT alone did not allow for clear identification ofthe factors affecting these implementation processes orhow the intervention was delivered, which are importantfor the purposes of a formative evaluation. Clarke et al.reported a similar finding when reflecting on their experi-ence of using NPT in a process evaluation of the TrainingCaregivers after Stroke (TRACS) Trial [38]. NPT was saidto place undue emphasis on individual and collectiveagency without acknowledging contextual factors thatimpacted on this agency. An unexpected finding whenusing NPT was therapists’ difficulty in trying to recallthe processes involved in implementing GRASP at theirwork site. Indeed, even the practice leaders who often initi-ated the implementation of GRASP struggled to recallwhen they had first heard about the intervention. Thera-pists could often remember only the processes in whichthey as individuals were directly involved e.g., a therapistwould presume that it was the rehabilitation assistant thatrestocked the GRASP equipment box, but would reportthat they were not entirely certain. This fits with edu-cational theories on learning in the workplace, whichhave found that ‘in everyday practices, learning takesplace in the flow of experience, with or without ourawareness of it’ [39].Interviewing therapists in this study, as opposed tousing audit or survey methodology, allowed us to getdeeper insight into implementation fidelity and the rea-sons for adaptations to the intervention as opposed to justthe way in which was adapted. The CFIF provided a com-prehensive structure for reporting the use of the GRASPthat will enable greater comparison across settings in thefuture. Domains from the CFIR were used to explain theresearch findings. The CFIR has been used previouslyusing a ‘menu of constructs approach’ [40], where thefocus has been on those factors relevant to the context ofinterest. When evaluating the implementation of a weightmanagement program, using a cross-case comparison ofratings, ten CFIR constructs strongly differentiated thelow versus high implementation facilities. The factorsidentified had parallels with our study, where networksand communications, leadership engagement, and relativeadvantage of the intervention were all found to influenceimplementation. This highlights the value of using such aframework, as consistent definitions allow for compari-sons and synthesis of findings across studies.Overlap between constructs was an issue, but the frame-works were not seen as mutually exclusive. Difficultieswith commonality have been discussed by others, and ithas been argued that irrespective of their coding, the useof frameworks helps to highlight important issues [17].The flexible use of NPT has been applauded, as it demon-strates critical use of the constructs, rather than a ‘concep-tual straitjacket’ [17,41]. We found the frameworks to beuseful in ensuring comprehensiveness, but used them ascomplementary rather than restrictive guides to exploredifferent aspects of the complex implementation elements.However, we did find the three frameworks were designedto be used from the perspective of the service providerwithout explicit consideration of the service-users. This isof particular importance for exploring implementationfidelity as characteristics of the recipients of interventionscan often provide valid reasons for adaptations to inter-ventions [29].There is currently no gold standard or agreement asto which frameworks should be used, with a plethoraConnell et al. Implementation Science 2014, 9:90 Page 10 of 12http://www.implementationscience.com/content/9/1/90available (e.g., Flottorp et al. identified 12 frameworksand 57 factors that explored determinants of practicealone [42]). With existing frameworks being continu-ally developed and refined, and the rapidly changingand advancing landscape of implementation science,there is as yet no clear guidance as to which frameworkto use and how it should be applied. Therefore we feelour approach was thorough and will inform the futureintervention development.LimitationsThe self-report data collected in this study relied ontherapists’ ability to recall events from a few months to acouple of years prior to the interviews. Therefore onemust be cautious when considering both the accuracyand the detail of these accounts. As participants inthis study were volunteers, a self-selection bias existswhere perhaps therapists with stronger opinions onthe programme and/or its implementation are overrepresented thus limiting the generalisability of thestudy findings. The self-report nature of the data alsointroduces the risk of a social desirability bias whereparticipants may have felt obliged to answer questionsin a way that would be deemed pleasing to the researchersconducting the interviews. However, the research teamboth prior to, and during the interviews highlighted toparticipants that the data collected would be anonymised,and that it would not be possible for them to be identifiedin the hope that they would be as candid as possible.Practice implications and future researchTo facilitate translation of effective interventions intoroutine clinical practice, it is of value to identify existingnetworks through which detailed information on how toimplement the intervention can be communicated. Freeonline access to this information, in the case of GRASP,has been found to be highly effective. Implementationfidelity is moderated by providers’ beliefs or concernsabout interventions. Co-creation of interventions duringdevelopment, ensuring they are evidence-based but alsobest-fit to the providers’ beliefs and context, may helpwith implementation and fidelity. In addition to theintervention content, a behaviour change element andimplementation strategy that facilitates the change inpractice warrants further research. There is an urgentneed for researchers to empirically test the ‘active ingredi-ents’ of package interventions so that the mechanisms ofaction can be communicated to those responsible for theirimplementation. It is known that adaptability of interven-tions facilitates implementation. Therefore creative solu-tions that allow adaptation of intervention componentswhile still delivering the active ingredients of interventionsare required. As non-adopters of GRASP were not includedin this study, we can only hypothesise possible reasons asto why this evidence-based intervention has yet to beimplemented in more stroke rehabilitation units. Futureresearch that objectively assesses actual uptake of inter-ventions and explores factors influencing non-adoptionof evidence-based interventions would provide furthervaluable information as to how interventions can be de-signed and adapted to improve congruence with therapistsacross settings.ConclusionThis study is particularly novel as data collection wasnot nested within a larger scale study to evaluate theintervention effectiveness. Instead, therapists and organi-sations in this region independently chose to adopt andimplement this intervention, and have continued to useit long after the original research study was completed.This opportunity to explore what one could describe as‘natural’ implementation of an intervention has provideda unique insight into how interventions translate fromresearch trials into routine clinical practice. AlthoughGRASP has translated into clinical practice, it is notused in the way in which it was shown to be effective.Novel therapist behaviour change interventions that areunderpinned by theory may improve the implementationand fidelity of interventions to facilitate evidence-basedpractice. This formative evaluation has informed thedevelopment of a novel intervention that aims to bridgethis evidence-practice gap in upper limb rehabilitationafter stroke.Additional filesAdditional file 1: Interview guide. Interview guide developed for datacollection (word document).Additional file 2: Coding frame. Coding frame demonstrating how thethree frameworks were used during data analysis (word document).Additional file 3: Recruitment of participants. Breakdown ofrecruitment of participants across sites contacted (word document).Competing interestsJE and JH developed GRASP but do not benefit financially in any way fromits use in clinical practice.Authors’ contributionsThe initial research proposal was developed by LC with input from CWand JE. Data collection and data analysis was carried out by LC and NM.All authors contributed to the write up of the findings and criticallyreviewing the final version for publication. All authors read andapproved the final manuscript.AcknowledgementsWe would like to acknowledge the occupational therapists, physiotherapistsand rehabilitation assistants that gave up their time to take part in this studyand provide candid accounts of their experiences of using GRASP in practice.We also wish to acknowledge the staff of the Research Lab at G.F. StrongRehabilitation Centre in Vancouver for the assistance they provided inconducting this study and Dr Brigit Chesworth for her valuable input incompiling the findings for publication. Louise Connell is funded by aNational Institute for Health Research (NIHR) Career DevelopmentConnell et al. Implementation Science 2014, 9:90 Page 11 of 12http://www.implementationscience.com/content/9/1/90Fellowship. The views and opinions expressed by authors in this publicationare those of the authors and do not necessarily reflect those of the NHS orthe NIHR.Author details1Clinical Practice Research Unit, School of Health, University of CentralLancashire, Preston PR1 2HE, UK. 2School of Rehabilitation Science, McMasterUniversity, 1400 Main Street West, Hamilton, Ontario L8S 1C7, Canada.3Department of Physical Therapy, University of British Columbia, 212-2177Wesbrook Mall, Vancouver V6T 1Z3British Columbia, Canada.Received: 20 December 2013 Accepted: 27 June 2014Published: 12 August 2014References1. Van Peppen RPS, Kwakkel G, Wood-Dauphinee S, Hendriks HJM, Van derWees PJ, Dekker J: The impact of physical therapy on functional out-comes after stroke: what’s the evidence? Clin Rehabil 2004, 18:833–862.2. van der Lee JH, Snels IAK, Beckerman H, Lankhorst GJ, Wagenaar RC, BouterLM: Exercise therapy for arm function in stroke patients: a systematicreview of randomized controlled trials. Clin Rehabil 2001, 15:20–31.3. Langhorne P, Coupar F, Pollock A: Motor recovery after stroke: asystematic review. Lancet Neurol 2009, 8:741–754.4. Veerbeek JM, van Wegen E, van Peppen R, van der Wees PJ, Hendriks E,Rietberg M, Kwakkel G: What is the evidence for physical therapypoststroke? a systematic review and meta-analysis. PLoS One 2014, 9:e87987.5. Lang C, MacDonald J, Gnip C: Counting repetitions: an observationalstudy of outpatient therapy for people with hemiparesis post-stroke.J Neurol Phys Ther 2007, 31:3–10.6. Lang C, MacDonald J, Reisman D, Boyd L, Kimberley T, Schindler-Ivens S,Hornby T, Ross S, Scheets P: Observation of amounts of movementpractice provided during stroke rehabilitation. Arch Phys Med Rehabil2009, 90:1692–1698.7. Harris JE, Eng JJ, Miller WC, Dawson AS: A self-administered graded repetitiveArm supplementary program (GRASP) improves Arm function duringinpatient stroke rehabilitation a multi-site randomized controlled trial.Stroke 2009, 40:2123–2128.8. Lindsay MP, Gubitz G, Bayley M, Hill MD, Davies-Schinkel C, Singh S, PhillipsS: Canadian Best Practice Recommendations for Stroke Care (Update 2010). Onbehalf of the Canadian Stroke Strategy Best Practices and Standards WritingGroup. Ottawa, Ontario Canada: Canadian Stroke Network; 2010:109.9. Sites that use GRASP. [http://neurorehab.med.ubc.ca/grasp-downloads-2/sites-that-use-grasp/]10. Connell LA, McMahon NE, Watkins CL, Eng JJ: Therapists’ use of theGraded Repetitive Arm Supplementary Program (GRASP) intervention: apractice implementation survey study. Phys Ther 2014, 94:632–643.11. Stetler CB, Legro MW, Wallace CM, Bowman C, Guihan M, Hagedorn H,Kimmel B, Sharp ND, Smith JL: The role of formative evaluation inimplementation research and the QUERI experience. J Gen Intern Med2006, 21(2):S1–S8.12. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC:Fostering implementation of health services research findings intopractice: a consolidated framework for advancing implementationscience. Implement Sci 2009, 4:50.13. May C, Finch T: Implementing, embedding, and integrating practices: anoutline of normalization process theory. Sociology 2009, 43:535–554.14. Hasson H, Blomberg S, Duner A: Fidelity and moderating factors incomplex interventions: a case study of a continuum of care program forfrail elderly people in health and social care. Implement Sci 2012, 7:23.15. Fixsen D, Naoom S, Blase K, Friedman R, Wallace F: Implementation Research:A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis dela Parte Florida Mental Health Institute, The National ImplementationResearch Network; 2005:17.16. Hsieh HF, Shannon SE: Three approaches to qualitative content analysis.Qual Health Res 2005, 15:1277–1288.17. McEvoy R, Ballini L, Maltoni S, O’Donnell CA, Mair FS, MacFarlane A: Aqualitative systematic review of studies using the normalization processtheory to research implementation processes. Implement Sci 2014, 9:2.18. Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S: A conceptualframework for implementation fidelity. Implement Sci 2007, 2:40.19. Hasson H: Systematic evaluation of implementation fidelity of complexinterventions in health and social care. Implement Sci 2010, 5:67.20. Connell LA, McMahon NE, Eng JJ, Watkins CL: Prescribing upper limbexercises after stroke: A survey of current UK therapy practice. J RehabMed 2014, 46:212–218.21. McCluskey A, Vratsistas-Curto A, Schurr K: Barriers and enablers to implementingmultiple stroke guideline recommendations: a qualitative study. BMC HealthServ Res 2013, 13:323–333.22. Dannapfel P, Peolsson A, Nilsen P: What supports physiotherapists’ use ofresearch in clinical practice? A qualitative study in Sweden. Implement Sci2013, 8:31.23. Gabbay J, le May A: Evidence based guidelines or collectively constructed‘mindlines?’ - Ethnographic study of knowledge management in primarycare. BMJ 2004, 329:1013–1016A.24. McCormack B, Rycroft-Malone J, DeCorby K, Hutchinson A, Bucknall T, KentB, Schultz A, Snelgrove-Clarke E, Stetler C, Titler M, Wallin L, Wilson V: A real-ist review of interventions and strategies to promote evidence-informedhealthcare: a focus on change agency. Implement Sci 2013, 8:107.25. Farley K, Hanbury A, Thompson C: Gathering opinion leader data for atailored implementation intervention in secondary healthcare: arandomised trial. BMC Medical Res Methodol 2014, 14:38.26. Harn B, Parisi D, Stoolmiller M: Balancing fidelity with flexibility and Fit:What do we really know about fidelity of implementation in schools?.Except Child 2013, 79:181–193.27. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion ofinnovations in service organizations: Systematic review andrecommendations. Milbank Q 2004, 82:581–629.28. Kollen BJ, Lennon S, Lyons B, Wheatley-Smith L, Scheper M, Buurke JH, Half-ens J, Geurts AC, Kwakkel G: The effectiveness of the Bobath concept instroke rehabilitation what is the evidence? Stroke 2009, 40:e89–e97.29. Durlak JA: The importance of doing well in whatever you do: Acommentary on the special section, ‘Implementation research in earlychildhood education.’. Early Child Res Q 2010, 25:348–357.30. Baker CN, Kupersmidt JB, Voegler-Lee ME, Arnold DH, Willoughby MT:Predicting teacher participation in a classroom-based, integrated preventiveintervention for preschoolers. Early Child Res Q 2010, 25:270–283.31. Leon N, Lewin S, Mathews C: Implementing a provider-initiated testingand counselling (PITC) intervention in Cape town, South Africa: a processevaluation using the normalisation process model. Implement Sci 2013,8:97.32. Lloyd A, Joseph-Williams N, Edwards A, Rix A, Elwyn G: Patchy ‘coherence’: usingnormalization process theory to evaluate a multi-faceted shared decisionmaking implementation program (MAGIC). Implement Sci 2013, 8:102.33. Arditi C, Rege-Walther M, Wyatt JC, Durieux P, Burnand B: Computer-generatedreminders delivered on paper to healthcare professionals; effects onprofessional practice and health care outcomes. Cochrane Database Syst Rev2012, 12:CD001175.34. Giguere A, Legare F, Grimshaw J, Turcotte S, Fiander M, Grudniewicz A,Makosso-Kallyth S, Wolf FM, Farmer AP, Gagnon M-P: Printed educationalmaterials: effects on professional practice and healthcare outcomes.Cochrane Database Syst Rev 2012, 10:CD004398.35. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD,O’Brien MA, Johansen M, Grimshaw J, Oxman AD: Audit and feedback:effects on professional practice and healthcare outcomes. CochraneDatabase Syst Rev 2012, 6:CD000259.36. Forsetlund L, Bjorndal A, Rashidian A, Jamtvedt G, O’Brien MA, Wolf F, DavisD, Odgaard-Jensen J, Oxman AD: Continuing education meetings andworkshops: effects on professional practice and health care outcomes.Cochrane Database Syst Rev 2009, 2:CD003030.37. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, Grimshaw JM: Nomore ‘business as usual’with audit and feedback interventions: towards anagenda for a reinvigorated intervention. Implement Sci 2014, 9:14.38. Clarke DJ, Godfrey M, Hawkins R, Sadler E, Harding G, Forster A, McKevitt C,Dickerson J, Farrin A: Implementing a training intervention to supportcaregivers after stroke: a process evaluation examining the initiation andembedding of programme change. Implement Sci 2013, 8:96.39. Gherardi S: Practice-based theorizing on learning and knowing inorganizations. Organization 2000, 7:211–223.40. Damschroder LJ, Lowery JC: Evaluation of a large-scale weight managementprogram using the consolidated framework for implementation research(CFIR). Implement Sci 2013, 8:51.41. Normalization process theory on-line Users’ manual and toolkit.[http://www.normalizationprocess.org]42. Flottorp SA, Oxman AD, Krause J, Musila NR, Wensing M, Godycki-Cwirko M,Baker R, Eccles MP: A checklist for identifying determinants of practice: Asystematic review and synthesis of frameworks and taxonomies offactors that prevent or enable improvements in healthcare professionalpractice. Implement Sci 2013, 8:35.doi:10.1186/s13012-014-0090-3Cite this article as: Connell et al.: A formative evaluation of theimplementation of an upper limb stroke rehabilitation intervention inclinical practice: a qualitative interview study. Implementation Science2014 9:90.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionConnell et al. Implementation Science 2014, 9:90 Page 12 of 12http://www.implementationscience.com/content/9/1/90Submit your manuscript at www.biomedcentral.com/submit


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items