UBC Library and Archives

The Realist Review Process Workshop Wong, Geoff; MacPhee, Maura; Merrett, Katelyn; Miller, Katherine; Taylor, Sally; Pawliuk, Colleen 2020-03-10

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Notice for Google Chrome users:
If you are having trouble viewing or searching the PDF with Google Chrome, please download it here instead.

Item Metadata

Download

Media
67246-Realist_Review_Process_Workshop_Part1.mp4 [ 1.8GB ]
67246-Realist_Review_Process_Workshop_Part2.mp4 [ 1.69GB ]
67246-MacPhee_M_et_al_Realist_Review_2020_slides.pdf [ 667.7kB ]
67246-MacPhee_M_et_al_Realist_Review_2020_handout.pdf [ 1.88MB ]
67246-MacPhee_M_et_al_Healthcare_Quarterly_Tables_2020.pdf [ 103.62kB ]
Metadata
JSON: 67246-1.0390457.json
JSON-LD: 67246-1.0390457-ld.json
RDF/XML (Pretty): 67246-1.0390457-rdf.xml
RDF/JSON: 67246-1.0390457-rdf.json
Turtle: 67246-1.0390457-turtle.txt
N-Triples: 67246-1.0390457-rdf-ntriples.txt
Original Record: 67246-1.0390457-source.json
Full Text
67246-1.0390457-fulltext.txt
Citation
67246-1.0390457.ris

Full Text

Background• The shift in health care delivery from acute care to community care• The shift from solo practitioner to team-based careOur Research Team• Geoff (Primary care physician, health services researcher, realist methodologist)• Maura (Nurse, health services researcher, program evaluation, qual/quant)• Ruth Abrams (psychologist, research associate-realist methods)• Katelyn Merrett (nurse, graduate student)Our Research QuestionsWhat team processes are associated with team effectiveness in Canadian primary healthcare clinic settings? What mechanisms and contextual factors result in team effectiveness? Study DesignReview design: Pawson et al. (2005)Realist review—a new method of systematic review designed for complex policy interventionsThe realist approach: “Causation is a generative process—where outcomes are caused by context sensitive mechanisms” (Abrams et al. 2018, p. 3)Stakeholder engagement• The realist review employs iterative cycles of stakeholder engagement.• Who is a relevant stakeholder?• How do you locate them? • When should you engage with them?• What can they do for you?Step 1. Locating existing theoriesStep 1. Locating existing theories: An iterative process• Stakeholders, subject matter experts• Exploratory search of the literature• Project team discussions• End result: An initial program theoryTeam Processes-An Initial Program TheoryStep 2. Searching for Evidence• You need a medical librarian. Many thanks to Katherine Miller.• You need a skilled research assistant. Many thanks to Katelyn Merrett.• Our search topics: Team processes in primary care contexts in Canada• Inclusion criteria: English, last 20 years, key search terms• Three databases:• CINAHL, Ovid Medline, TRIP database• Over 4,000      120• Title, abstract, key words: 10%  independent consistency checkSearch StrategyTable 2  MEDLINE (Ovid) Search Strategy Search Term Concepts Results from Search Results: Terms Combined with “OR” Results: 3 Search Concepts Combined with AND    TEAM SH: Patient Care Team OR Nursing Team  SH: Interprofessional Relations OR Interdisciplinary Communication OR Physician-Nurse Relations  (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidiscipin* OR Intraprofession*).ti,ab,kw. 64282  66333     286372     367908    1095   Results limited to English Language: 1062       PRIMARY HEALTH CARE SH: Primary Health Care/ (includes all subheadings)  (Primary Health Care OR Primary Healthcare OR Primary Care).ti,ab,kw. 70749   126022  147102   CANADA SH: Canada/ OR Alberta/ OR British Columbia/ OR Manitoba/ OR New Brunswick/ OR “Newfoundland and Labrador”/ OR Northwest Territories/ OR Nova Scotia/ Or Nunavut/ OR Ontario/ OR Prince Edward Island/ OR Quebec/ OR Saskatchewan/ OR Yukon Territory/   (Canad* OR Newfoundland OR Prince Edward Island OR Nova Scotia OR New Brunswick OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR British Columbia OR Yukon OR Northwest Territories”OR Nunavut).ti,ab,kw.     148063      149922        216673   Note. All search results from February 7, 2019.  Table 1  CINAHL Search Strategy Search Term Concepts Results from Search Results: Terms Combined with “OR” Results: 3 Search Concepts Combined with AND    TEAM (MH (medical heading): Teamwork OR MH: “Multidisciplinary Care Team”)  (MH: Collaboration OR MH: “Interprofessional Relations” OR MH: “Intraprofessional Relations” OR MH: “Nurse-Physician Relations”)  TI (title): (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*)  AB (abstract): (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*) 49,438    66,034    51,074    135,025       230,378                     834 PRIMARY HEALTH CARE MH: “Primary Health Care”  TI: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  [TI: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare” OR "Family Health Team*")]  AB: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  [AB: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare” OR "Family Health Team*")] 53,621   31,283   31,480   49,725   50,102    88,838   [with “Family Health Team*”] 89,284 CANADA MH: Canada+  TI: (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut)  AB: (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut) 88,978    29,522       45,468       112,237 Note. All results from February 7, 2019.  Search Strategy → Grey Literature Table 3  TRIP Database Search Strategy Search Term Concepts Results from Search Results: 3 Search Concepts Combined with AND Added Specifiers    TEAM (Team* OR Collaborat* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*)   142,445       9,558 Primary Care/Family Practice: 549   Searched (Team* AND "Primary Care" AND Canada) and found 63 additional sources.  No unique results when ‘Interprofessiona’ is used instead of ‘Team’ / No unique results when “Primary Health Care” used instead of “Primary Care”  Added: 2 grey literature sources from Grelit.org  TOTAL: 614 PRIMARY HEALTH CARE (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  88,725 CANADA (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut)     257,654 Note. Search Results from Friday, February 8, 2019. PRISMA ChartStep 3. Document Selection• Documents are selected based on:• Relevance• RigourRelevance=contribution to theory development and refinementRigour=credibility and trustworthiness of the methods used to generate data • Consistency checks by two reviewers on 10% of documents  Step 4. Data ExtractionTwo-fold data extraction:1. Table of documents with descriptive information2. Document upload for data analysis (NVivo 12)Step 5. Data Analysis• Data analysis uses “realist logic analysis” to refine the program theory• A unique aspect of realist data analysis:• Always looking for potential linkages context-mechanism-outcome configurations (CMOCs)• Our data coding was:• Deductive (based on initial program theory)• Inductive (emerging from the documents)• Retroductive (potential causal processes or mechanisms)• We conducted consistency checks on 10% of coded documentsStep 6. Program Theory Refinement• Discuss, refine, hone the final theory with its CMOCs• The final program theory/CMOCs is testable through realist evaluation • Our realist evaluation: In partnership with Pender ClinicCMOCs# Forming: Establishing social ground rules, learning about each other’s scope and roles, creating a psychologically safe workp lace1 CMOC: When team members have opportunities to interact with each other (C), perspective-taking occurs (M), laying the foundation for social bonds and trusting relationships (O).2 CMOC:  When teams have dedicated supports in place (C),  team members have increased opportunities for perspective-taking (M), resulting in more awareness of each other’s roles and scopes (O)3 CMOC: When team members are aware, interested and concerned about others (C), they strive to demonstrate carefulness in their actions and words (M), resulting in greater workplace psychological safety (O)Storming: Working out conflict, narrowing the us-them gap4 CMOC: When team members lack carefulness for others (C), there is decreased perspective-taking (M), resulting in more conflict, less willingness to work collaboratively, and a greater us-them gap (O). Norming: Developing deeper, affective trust and a collective team identity5 CMOC: When team members have planned time and space for ongoing opportunities to interact with each other (C), because of perspective-taking (M), they begin developing affective trust in one another (O). 6 CMOC: When team members openly share information and ideas (C), because of affective trust (M), they develop a collective team identity (O).Performing: Managing ambiguity, inviting change and innovation7 CMOC: When team members use carefulness and perspective-taking (C), because of deep trust (M) they are comfortable with ambiguity and have the capacity to champion innovation (O). Program TheoryFigure 1. Program theory for Team Processes in Canadian Primary Healthcare Settings  Forming                                                      Storming  Norming   Performing Establishing social ground rules  Working out conflict Co-creating new norms          Managing ambiguity Learning each other’s scope, roles     Forging a collective identity       Inviting innovation         Sharing critical information         Sharing decisions        Leadership Knowledge TranslationMechanism StrategiesPerspective-taking • Routinized, frequent interactions• Briefing, huddles, debriefing• Team meetingsCarefulness Relational education• Mindfulness• Appreciative Inquiry• Reflected Best SelfTrust • Formal, facilitated orientation• Creation of team ground rules• Shadow experiencesTable 2. Evidence-Informed Strategies for Key Team Processes Mechanisms Background• The shift in health care delivery from acute care to community care• The shift from solo practitioner to team-based careOur Research Team• Geoff (Primary care physician, health services researcher, realist methodologist)• Maura (Nurse, health services researcher, program evaluation, qual/quant)• Ruth Abrams (psychologist, research associate-realist methods)• Katelyn Merrett (nurse, graduate student)Our Research QuestionsWhat team processes are associated with team effectiveness in Canadian primary healthcare clinic settings? What mechanisms and contextual factors result in team effectiveness? Study DesignReview design: Pawson et al. (2005)Realist review—a new method of systematic review designed for complex policy interventionsThe realist approach: “Causation is a generative process—where outcomes are caused by context sensitive mechanisms” (Abrams et al. 2018, p. 3)Stakeholder engagement• The realist review employs iterative cycles of stakeholder engagement.• Who is a relevant stakeholder?• How do you locate them? • When should you engage with them?• What can they do for you?Step 1. Locating existing theoriesStep 1. Locating existing theories: An iterative process• Stakeholders, subject matter experts• Exploratory search of the literature• Project team discussions• End result: An initial program theoryTeam Processes-An Initial Program TheoryStep 2. Searching for Evidence• You need a medical librarian. Many thanks to Katherine Miller.• You need a skilled research assistant. Many thanks to Katelyn Merrett.• Our search topics: Team processes in primary care contexts in Canada• Inclusion criteria: English, last 20 years, key search terms• Three databases:• CINAHL, Ovid Medline, TRIP database• Over 4,000      120• Title, abstract, key words: 10%  independent consistency checkSearch StrategyTable 2  MEDLINE (Ovid) Search Strategy Search Term Concepts Results from Search Results: Terms Combined with “OR” Results: 3 Search Concepts Combined with AND    TEAM SH: Patient Care Team OR Nursing Team  SH: Interprofessional Relations OR Interdisciplinary Communication OR Physician-Nurse Relations  (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidiscipin* OR Intraprofession*).ti,ab,kw. 64282  66333     286372     367908    1095   Results limited to English Language: 1062       PRIMARY HEALTH CARE SH: Primary Health Care/ (includes all subheadings)  (Primary Health Care OR Primary Healthcare OR Primary Care).ti,ab,kw. 70749   126022  147102   CANADA SH: Canada/ OR Alberta/ OR British Columbia/ OR Manitoba/ OR New Brunswick/ OR “Newfoundland and Labrador”/ OR Northwest Territories/ OR Nova Scotia/ Or Nunavut/ OR Ontario/ OR Prince Edward Island/ OR Quebec/ OR Saskatchewan/ OR Yukon Territory/   (Canad* OR Newfoundland OR Prince Edward Island OR Nova Scotia OR New Brunswick OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR British Columbia OR Yukon OR Northwest Territories”OR Nunavut).ti,ab,kw.     148063      149922        216673   Note. All search results from February 7, 2019.  Table 1  CINAHL Search Strategy Search Term Concepts Results from Search Results: Terms Combined with “OR” Results: 3 Search Concepts Combined with AND    TEAM (MH (medical heading): Teamwork OR MH: “Multidisciplinary Care Team”)  (MH: Collaboration OR MH: “Interprofessional Relations” OR MH: “Intraprofessional Relations” OR MH: “Nurse-Physician Relations”)  TI (title): (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*)  AB (abstract): (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*) 49,438    66,034    51,074    135,025       230,378                     834 PRIMARY HEALTH CARE MH: “Primary Health Care”  TI: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  [TI: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare” OR "Family Health Team*")]  AB: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  [AB: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare” OR "Family Health Team*")] 53,621   31,283   31,480   49,725   50,102    88,838   [with “Family Health Team*”] 89,284 CANADA MH: Canada+  TI: (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut)  AB: (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut) 88,978    29,522       45,468       112,237 Note. All results from February 7, 2019.  Search Strategy → Grey Literature Table 3  TRIP Database Search Strategy Search Term Concepts Results from Search Results: 3 Search Concepts Combined with AND Added Specifiers    TEAM (Team* OR Collaborat* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*)   142,445       9,558 Primary Care/Family Practice: 549   Searched (Team* AND "Primary Care" AND Canada) and found 63 additional sources.  No unique results when ‘Interprofessiona’ is used instead of ‘Team’ / No unique results when “Primary Health Care” used instead of “Primary Care”  Added: 2 grey literature sources from Grelit.org  TOTAL: 614 PRIMARY HEALTH CARE (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  88,725 CANADA (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut)     257,654 Note. Search Results from Friday, February 8, 2019. PRISMA ChartStep 3. Document Selection• Documents are selected based on:• Relevance• RigourRelevance=contribution to theory development and refinementRigour=credibility and trustworthiness of the methods used to generate data • Consistency checks by two reviewers on 10% of documents  Step 4. Data ExtractionTwo-fold data extraction:1. Table of documents with descriptive information2. Document upload for data analysis (NVivo 12)Step 5. Data Analysis• Data analysis uses “realist logic analysis” to refine the program theory• A unique aspect of realist data analysis:• Always looking for potential linkages context-mechanism-outcome configurations (CMOCs)• Our data coding was:• Deductive (based on initial program theory)• Inductive (emerging from the documents)• Retroductive (potential causal processes or mechanisms)• We conducted consistency checks on 10% of coded documentsStep 6. Program Theory Refinement• Discuss, refine, hone the final theory with its CMOCs• The final program theory/CMOCs is testable through realist evaluation • Our realist evaluation: In partnership with Pender ClinicCMOCs# Forming: Establishing social ground rules, learning about each other’s scope and roles, creating a psychologically safe workp lace1 CMOC: When team members have opportunities to interact with each other (C), perspective-taking occurs (M), laying the foundation for social bonds and trusting relationships (O).2 CMOC:  When teams have dedicated supports in place (C),  team members have increased opportunities for perspective-taking (M), resulting in more awareness of each other’s roles and scopes (O)3 CMOC: When team members are aware, interested and concerned about others (C), they strive to demonstrate carefulness in their actions and words (M), resulting in greater workplace psychological safety (O)Storming: Working out conflict, narrowing the us-them gap4 CMOC: When team members lack carefulness for others (C), there is decreased perspective-taking (M), resulting in more conflict, less willingness to work collaboratively, and a greater us-them gap (O). Norming: Developing deeper, affective trust and a collective team identity5 CMOC: When team members have planned time and space for ongoing opportunities to interact with each other (C), because of perspective-taking (M), they begin developing affective trust in one another (O). 6 CMOC: When team members openly share information and ideas (C), because of affective trust (M), they develop a collective team identity (O).Performing: Managing ambiguity, inviting change and innovation7 CMOC: When team members use carefulness and perspective-taking (C), because of deep trust (M) they are comfortable with ambiguity and have the capacity to champion innovation (O). Program TheoryFigure 1. Program theory for Team Processes in Canadian Primary Healthcare Settings  Forming                                                      Storming  Norming   Performing Establishing social ground rules  Working out conflict Co-creating new norms          Managing ambiguity Learning each other’s scope, roles     Forging a collective identity       Inviting innovation         Sharing critical information         Sharing decisions        Leadership Knowledge TranslationMechanism StrategiesPerspective-taking • Routinized, frequent interactions• Briefing, huddles, debriefing• Team meetingsCarefulness Relational education• Mindfulness• Appreciative Inquiry• Reflected Best SelfTrust • Formal, facilitated orientation• Creation of team ground rules• Shadow experiencesTable 2. Evidence-Informed Strategies for Key Team Processes Mechanisms Realist Reviews / SynthesesDr Geoff WongClinical Reearch Fellowgeoffrey.wong@phc.ox.ac.ukTuesday 10th March 2020Hosted by UBC Library and the Health Libraries Association of BCRAMESES projects: www.ramesesproject.orgRAMESES on JISCM@il: www.jiscmail.ac.uk/RAMESESUniversity of Oxford training course on Realist Review and Realist Evaluation (RR&RE) – more details: goo.gl/4vuqC8 Acknowledgements and disclaimers• Geoff Wong’s salary is partly supported by The Evidence Synthesis Working Group of the National Institute for Health Research School for Primary Care Research (NIHR SPCR) [Project Number 390].• The views expressed are those of the author(s) and not necessarily those of the NIHR, the NHS or the Department of Health.• Geoff Wong has no conflicts of interest to declare.Structure of this presentation• Why use a realist review approach?• What is realism?• The underpinning assumptions of realist research• Brief overview of realist review• QuestionsProgrammes or interventions in health are complexhttp://crucesydesplazamientos.files.wordpress.com/2008/05/liverpoolmap.jpgMultiple componentsMultiple outcomesLong implementation chainsNon-linearInteracting componentsEmergenceContext sensitive outcomesA simplified diagram of a ‘complex’ interventionInterventione.g. a smoking cessation programme1944 - 2014REALISMContext + Mechanism = OutcomeA way of thinking about:• how the world is constituted• causation• theories• data• why lessons are transferableClarifying concepts …• Context– The ‘things’, ‘world’, ‘structures’ around a person• Mechanism– “…underlying entities, processes, or structures which operate in particular contexts to generate outcomes of interest.”*– The way in which a programme’s resources or opportunities interact with the reasoning of individuals and lead to changes in behaviour.The reasoning may or may not be ‘rational’!– Usually hidden– Sensitive to variations in context– Generate outcomes• Outcomes– May or many not be observable!• Context-Mechanism-Outcome Configurations (CMOCs)– Context, Mechanisms and Outcomes do not ‘free-float’– Context and Mechanism are linked to an Outcome*Astbury B, Leeuw F. Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation American Journal of Evaluation 2010 31(3):363-381• “Middle-range theory involves abstraction, of course, but they are close enough to observed data to be incorporated in propositions that permit empirical testing.”Merton R. On Theoretical Sociology. Five Essays, Old and New. New York: The Free Press, 1967.• In simple terms, a theory that is at the correct level of abstraction to be ‘useful’ and ‘testable’.• Realist approaches (realist review and realist evaluation) have a specific way of expressing a middle-range theory:Context (C) + Mechanism (M) = Outcome (O)Clarifying concepts …Programme theory - an abstracted description and/or diagram that lays out what a programme (or family of programmes or intervention) comprises and how it is expected to workIntervention strategyC3+M2=O4C4+M3=O4C4+M3=O4Intervention strategyC3+M2=O3C4+M3=O3Intervention strategyC3+M2=O2C3+M1=O1Intervention strategyC3+M2=O2C3+M1=O2C1+M3=O3Intervention strategyC1+M1=O1C2+M1=O1Diagrammatic representation of a refined realist programme theoryClarifying concepts …What happens in a complex intervention?• When participants take part in a complex intervention, they make choices about what actions to undertake and these choices about actions give us our outcomes.• Participants do not have an infinite range of choices available to them as to what actions they might take.• The range of choices is limited anddetermined by the context in whichthe person is in.• Various ‘mechanisms’ drive thesechoices.CMO and middle-range theory• Hence…Context + Mechanism = Outcome• Mechanisms are one of the building blocks of middle-range theories.• Middle-range theories explain how and why the context limits and influences mechanisms.Context influenceswhich mechanisms‘fire’.InterventionPutting it all togetherInterventionContextMechanism+OutcomeTo recap …• Agency lies with us - our heads ‘contain’ various different mechanisms• We respond to the world around us - mechanism ‘fire’ or are ‘triggered’ by certain contexts• The world is ‘full of’ context - interventions try to change context so that the ‘right’ mechanisms are triggered• Patterns (or demi-regularities) of C + M = O occur(i.e. certain people tend to behave in certain ways under certain situations)When demi-regularities occur, one possible inference is that the samemechanisms may be causing the outcome• Middle-range theories explain the limitations and/or influence of context on mechanisms behind these demi-regularities.• The key goal of a realist reviews is to uncover these middle-range theories.Realist review: A type of theory driven systematic reviewRefined programme theory Search for evidenceDevelop, pilot and refine searchScreeningArticle selectionRelevanceRigourExtracting and organising dataSynthesising the evidence (and drawing conclusions)Iteratively develop and provide recommendationsStartReview questionsGeneral designDesigning the reviewFormulating the initial programme theoryADAPTED FROM: PROTOCOL - Interventions to improve antimicrobial prescribing of doctors in training: the IMPACT (IMProving Antimicrobial presCribing of doctors in Training) realist review (In press)* if necessary*Any questions?Thank you for listening and for your questionsSuggested readings:• Pawson R., Tilley N. Realistic Evaluation. London: Sage, 1999• Pawson R. Evidence-based Policy. A Realist Perspective. London: Sage, 2006.• Pawson R. The Science of evaluation: A realist manifesto. London: Sage, 2013• RAMESES on JISCM@il – www.jiscmail.ac.uk/RAMESES• The RAMESES Projects – www.ramesesproject.orgSupplementary slides – for information onlyThe supplementary slides cover:• Common pitfalls in realist reviews• Quality in the reporting and conduct of realist reviews• Three examples of when realist reviews have been used with references to the full text articlesCommon pitfalls in realist reviews• No programme theory.• Programme theory is not realist.• Not enough relevant data (e.g. exclusion by study type, single search, too tight inclusion criteria, not looked for documents where the same mechanism may be in operation).• Did a thematic analysis / did not apply a realist logic of analysis or used some other logic of analysis.• Confused intervention strategy with mechanism.• Unconfigured C, M, O (‘CMO soup’).Quality in reporting realist reviews• RAMESES I Project (2011-2013).• Reporting standards published:http://bmcmedicine.biomedcentral.com/articles/10.1186/1741-7015-11-21• 19-item checklist.• Has built in flexibility – acknowledging the variety in purpose and users’ needs of realist reviews.• Not all items need to be reported – but if an item is left out this should be highlighted and justified.Quality in execution – realist reviews• Quality standards available for peer-reviewers and also for funders.• 8 criteria and uses a rubric approach –inadequate/adequate/good/excellent.• Available on RAMESES Project website (www.ramesesproject.org) under Standards and Training materials.Realist review example – unpacking the problem• Review of interventions to improve the antimicrobial practice of doctors-in-training• Most interventions took a narrow focus on providing education• Outcomes were mixed or unclear• Unpacking revealed the stronger influence of hierachies in driving prescribing behaviour– Wanting to fit in– Following seniors– Reputation management• Papoutsi C, Mattick K, Pearson M, Brennan N, Briscoe S, Wong G. Social and professional influences on antimicrobial prescribing for doctors-in-training: a realist review. Journal of Antimicrobial Chemotherapy 2017; dkx194. doi: 10.1093/jac/dkx194. Realist review example – developing interventions• Review to develop interventional strategies to improve access to primary care for socioeconomically disadvantaged older people in rural areas• Programme theory of patient journey• Each stage’s outcome has a CMO configuration• Ford J, Wong G, Jones A, Steel N. Access to primary care for socioeconomically disadvantaged older people in rural areas: a realist review. BMJ Open 2016; 6:e010652. Realist review example – developing interventionsRealist review example – transferability of mechanisms• Review to develop a programme theory to understand if a piece of public health legislation was likely to be successfully implemented• Programme theory tested in a ‘desktop’ exercise on legislation banning smoking in vehicles carrying children• In 2010, no evaluation of such legislation existed• For the enforcement part of the programme theory we had to extrapolate from car seat legislation• Wong G, Pawson R, Owen L. Policy guidance on threats to legislative interventions in public health: a realist synthesis. BMC Public Health 2011; 11(222) • Pawson R, Wong G, Owen L. Known Knowns, Known Unknowns, Unknown Unknowns: The Predicament of Evidence-Based Policy. American Journal of Evaluation 2011; 32(4):518-546  Background• The shift in health care delivery from acute care to community care• The shift from solo practitioner to team-based careOur Research Team• Geoff (Primary care physician, health services researcher, realist methodologist)• Maura (Nurse, health services researcher, program evaluation, qual/quant)• Ruth Abrams (psychologist, research associate-realist methods)• Katelyn Merrett (nurse, graduate student)Our Research QuestionsWhat team processes are associated with team effectiveness in Canadian primary healthcare clinic settings? What mechanisms and contextual factors result in team effectiveness? Study DesignReview design: Pawson et al. (2005)Realist review—a new method of systematic review designed for complex policy interventionsThe realist approach: “Causation is a generative process—where outcomes are caused by context sensitive mechanisms” (Abrams et al. 2018, p. 3)Stakeholder engagement• The realist review employs iterative cycles of stakeholder engagement.• Who is a relevant stakeholder?• How do you locate them? • When should you engage with them?• What can they do for you?Step 1. Locating existing theoriesStep 1. Locating existing theories: An iterative process• Stakeholders, subject matter experts• Exploratory search of the literature• Project team discussions• End result: An initial program theoryTeam Processes-An Initial Program TheoryStep 2. Searching for Evidence• You need a medical librarian. Many thanks to Katherine Miller.• You need a skilled research assistant. Many thanks to Katelyn Merrett.• Our search topics: Team processes in primary care contexts in Canada• Inclusion criteria: English, last 20 years, key search terms• Three databases:• CINAHL, Ovid Medline, TRIP database• Over 4,000      120• Title, abstract, key words: 10%  independent consistency checkSearch StrategyTable 2  MEDLINE (Ovid) Search Strategy Search Term Concepts Results from Search Results: Terms Combined with “OR” Results: 3 Search Concepts Combined with AND    TEAM SH: Patient Care Team OR Nursing Team  SH: Interprofessional Relations OR Interdisciplinary Communication OR Physician-Nurse Relations  (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidiscipin* OR Intraprofession*).ti,ab,kw. 64282  66333     286372     367908    1095   Results limited to English Language: 1062       PRIMARY HEALTH CARE SH: Primary Health Care/ (includes all subheadings)  (Primary Health Care OR Primary Healthcare OR Primary Care).ti,ab,kw. 70749   126022  147102   CANADA SH: Canada/ OR Alberta/ OR British Columbia/ OR Manitoba/ OR New Brunswick/ OR “Newfoundland and Labrador”/ OR Northwest Territories/ OR Nova Scotia/ Or Nunavut/ OR Ontario/ OR Prince Edward Island/ OR Quebec/ OR Saskatchewan/ OR Yukon Territory/   (Canad* OR Newfoundland OR Prince Edward Island OR Nova Scotia OR New Brunswick OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR British Columbia OR Yukon OR Northwest Territories”OR Nunavut).ti,ab,kw.     148063      149922        216673   Note. All search results from February 7, 2019.  Table 1  CINAHL Search Strategy Search Term Concepts Results from Search Results: Terms Combined with “OR” Results: 3 Search Concepts Combined with AND    TEAM (MH (medical heading): Teamwork OR MH: “Multidisciplinary Care Team”)  (MH: Collaboration OR MH: “Interprofessional Relations” OR MH: “Intraprofessional Relations” OR MH: “Nurse-Physician Relations”)  TI (title): (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*)  AB (abstract): (Collaborat* OR Team* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*) 49,438    66,034    51,074    135,025       230,378                     834 PRIMARY HEALTH CARE MH: “Primary Health Care”  TI: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  [TI: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare” OR "Family Health Team*")]  AB: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  [AB: (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare” OR "Family Health Team*")] 53,621   31,283   31,480   49,725   50,102    88,838   [with “Family Health Team*”] 89,284 CANADA MH: Canada+  TI: (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut)  AB: (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut) 88,978    29,522       45,468       112,237 Note. All results from February 7, 2019.  Search Strategy → Grey Literature Table 3  TRIP Database Search Strategy Search Term Concepts Results from Search Results: 3 Search Concepts Combined with AND Added Specifiers    TEAM (Team* OR Collaborat* OR Interprofession* OR Interdisciplin* OR Multidisciplin* OR Intraprofession*)   142,445       9,558 Primary Care/Family Practice: 549   Searched (Team* AND "Primary Care" AND Canada) and found 63 additional sources.  No unique results when ‘Interprofessiona’ is used instead of ‘Team’ / No unique results when “Primary Health Care” used instead of “Primary Care”  Added: 2 grey literature sources from Grelit.org  TOTAL: 614 PRIMARY HEALTH CARE (“Primary Care” OR “Primary Health Care” OR “Primary Healthcare”)  88,725 CANADA (Canad* OR Newfoundland OR Prince Edward Island OR “Nova Scotia” OR “New Brunswick” OR Quebec OR Ontario OR Manitoba OR Saskatchewan OR Alberta OR “British Columbia” OR Yukon OR “Northwest Territories” OR Nunavut)     257,654 Note. Search Results from Friday, February 8, 2019. PRISMA ChartStep 3. Document Selection• Documents are selected based on:• Relevance• RigourRelevance=contribution to theory development and refinementRigour=credibility and trustworthiness of the methods used to generate data • Consistency checks by two reviewers on 10% of documents  Step 4. Data ExtractionTwo-fold data extraction:1. Table of documents with descriptive information2. Document upload for data analysis (NVivo 12)Step 5. Data Analysis• Data analysis uses “realist logic analysis” to refine the program theory• A unique aspect of realist data analysis:• Always looking for potential linkages context-mechanism-outcome configurations (CMOCs)• Our data coding was:• Deductive (based on initial program theory)• Inductive (emerging from the documents)• Retroductive (potential causal processes or mechanisms)• We conducted consistency checks on 10% of coded documentsStep 6. Program Theory Refinement• Discuss, refine, hone the final theory with its CMOCs• The final program theory/CMOCs is testable through realist evaluation • Our realist evaluation: In partnership with Pender ClinicCMOCs# Forming: Establishing social ground rules, learning about each other’s scope and roles, creating a psychologically safe workp lace1 CMOC: When team members have opportunities to interact with each other (C), perspective-taking occurs (M), laying the foundation for social bonds and trusting relationships (O).2 CMOC:  When teams have dedicated supports in place (C),  team members have increased opportunities for perspective-taking (M), resulting in more awareness of each other’s roles and scopes (O)3 CMOC: When team members are aware, interested and concerned about others (C), they strive to demonstrate carefulness in their actions and words (M), resulting in greater workplace psychological safety (O)Storming: Working out conflict, narrowing the us-them gap4 CMOC: When team members lack carefulness for others (C), there is decreased perspective-taking (M), resulting in more conflict, less willingness to work collaboratively, and a greater us-them gap (O). Norming: Developing deeper, affective trust and a collective team identity5 CMOC: When team members have planned time and space for ongoing opportunities to interact with each other (C), because of perspective-taking (M), they begin developing affective trust in one another (O). 6 CMOC: When team members openly share information and ideas (C), because of affective trust (M), they develop a collective team identity (O).Performing: Managing ambiguity, inviting change and innovation7 CMOC: When team members use carefulness and perspective-taking (C), because of deep trust (M) they are comfortable with ambiguity and have the capacity to champion innovation (O). Program TheoryFigure 1. Program theory for Team Processes in Canadian Primary Healthcare Settings  Forming                                                      Storming  Norming   Performing Establishing social ground rules  Working out conflict Co-creating new norms          Managing ambiguity Learning each other’s scope, roles     Forging a collective identity       Inviting innovation         Sharing critical information         Sharing decisions        Leadership Knowledge TranslationMechanism StrategiesPerspective-taking • Routinized, frequent interactions• Briefing, huddles, debriefing• Team meetingsCarefulness Relational education• Mindfulness• Appreciative Inquiry• Reflected Best SelfTrust • Formal, facilitated orientation• Creation of team ground rules• Shadow experiencesTable 2. Evidence-Informed Strategies for Key Team Processes Mechanisms Realist Reviews / SynthesesDr Geoff WongClinical Reearch Fellowgeoffrey.wong@phc.ox.ac.ukTuesday 10th March 2020Hosted by UBC Library and the Health Libraries Association of BCRAMESES projects: www.ramesesproject.orgRAMESES on JISCM@il: www.jiscmail.ac.uk/RAMESESUniversity of Oxford training course on Realist Review and Realist Evaluation (RR&RE) – more details: goo.gl/4vuqC8 Acknowledgements and disclaimers• Geoff Wong’s salary is partly supported by The Evidence Synthesis Working Group of the National Institute for Health Research School for Primary Care Research (NIHR SPCR) [Project Number 390].• The views expressed are those of the author(s) and not necessarily those of the NIHR, the NHS or the Department of Health.• Geoff Wong has no conflicts of interest to declare.Structure of this presentation• Why use a realist review approach?• What is realism?• The underpinning assumptions of realist research• Brief overview of realist review• QuestionsProgrammes or interventions in health are complexhttp://crucesydesplazamientos.files.wordpress.com/2008/05/liverpoolmap.jpgMultiple componentsMultiple outcomesLong implementation chainsNon-linearInteracting componentsEmergenceContext sensitive outcomesA simplified diagram of a ‘complex’ interventionInterventione.g. a smoking cessation programme1944 - 2014REALISMContext + Mechanism = OutcomeA way of thinking about:• how the world is constituted• causation• theories• data• why lessons are transferableClarifying concepts …• Context– The ‘things’, ‘world’, ‘structures’ around a person• Mechanism– “…underlying entities, processes, or structures which operate in particular contexts to generate outcomes of interest.”*– The way in which a programme’s resources or opportunities interact with the reasoning of individuals and lead to changes in behaviour.The reasoning may or may not be ‘rational’!– Usually hidden– Sensitive to variations in context– Generate outcomes• Outcomes– May or many not be observable!• Context-Mechanism-Outcome Configurations (CMOCs)– Context, Mechanisms and Outcomes do not ‘free-float’– Context and Mechanism are linked to an Outcome*Astbury B, Leeuw F. Unpacking Black Boxes: Mechanisms and Theory Building in Evaluation American Journal of Evaluation 2010 31(3):363-381• “Middle-range theory involves abstraction, of course, but they are close enough to observed data to be incorporated in propositions that permit empirical testing.”Merton R. On Theoretical Sociology. Five Essays, Old and New. New York: The Free Press, 1967.• In simple terms, a theory that is at the correct level of abstraction to be ‘useful’ and ‘testable’.• Realist approaches (realist review and realist evaluation) have a specific way of expressing a middle-range theory:Context (C) + Mechanism (M) = Outcome (O)Clarifying concepts …Programme theory - an abstracted description and/or diagram that lays out what a programme (or family of programmes or intervention) comprises and how it is expected to workIntervention strategyC3+M2=O4C4+M3=O4C4+M3=O4Intervention strategyC3+M2=O3C4+M3=O3Intervention strategyC3+M2=O2C3+M1=O1Intervention strategyC3+M2=O2C3+M1=O2C1+M3=O3Intervention strategyC1+M1=O1C2+M1=O1Diagrammatic representation of a refined realist programme theoryClarifying concepts …What happens in a complex intervention?• When participants take part in a complex intervention, they make choices about what actions to undertake and these choices about actions give us our outcomes.• Participants do not have an infinite range of choices available to them as to what actions they might take.• The range of choices is limited anddetermined by the context in whichthe person is in.• Various ‘mechanisms’ drive thesechoices.CMO and middle-range theory• Hence…Context + Mechanism = Outcome• Mechanisms are one of the building blocks of middle-range theories.• Middle-range theories explain how and why the context limits and influences mechanisms.Context influenceswhich mechanisms‘fire’.InterventionPutting it all togetherInterventionContextMechanism+OutcomeTo recap …• Agency lies with us - our heads ‘contain’ various different mechanisms• We respond to the world around us - mechanism ‘fire’ or are ‘triggered’ by certain contexts• The world is ‘full of’ context - interventions try to change context so that the ‘right’ mechanisms are triggered• Patterns (or demi-regularities) of C + M = O occur(i.e. certain people tend to behave in certain ways under certain situations)When demi-regularities occur, one possible inference is that the samemechanisms may be causing the outcome• Middle-range theories explain the limitations and/or influence of context on mechanisms behind these demi-regularities.• The key goal of a realist reviews is to uncover these middle-range theories.Realist review: A type of theory driven systematic reviewRefined programme theory Search for evidenceDevelop, pilot and refine searchScreeningArticle selectionRelevanceRigourExtracting and organising dataSynthesising the evidence (and drawing conclusions)Iteratively develop and provide recommendationsStartReview questionsGeneral designDesigning the reviewFormulating the initial programme theoryADAPTED FROM: PROTOCOL - Interventions to improve antimicrobial prescribing of doctors in training: the IMPACT (IMProving Antimicrobial presCribing of doctors in Training) realist review (In press)* if necessary*Any questions?Thank you for listening and for your questionsSuggested readings:• Pawson R., Tilley N. Realistic Evaluation. London: Sage, 1999• Pawson R. Evidence-based Policy. A Realist Perspective. London: Sage, 2006.• Pawson R. The Science of evaluation: A realist manifesto. London: Sage, 2013• RAMESES on JISCM@il – www.jiscmail.ac.uk/RAMESES• The RAMESES Projects – www.ramesesproject.orgSupplementary slides – for information onlyThe supplementary slides cover:• Common pitfalls in realist reviews• Quality in the reporting and conduct of realist reviews• Three examples of when realist reviews have been used with references to the full text articlesCommon pitfalls in realist reviews• No programme theory.• Programme theory is not realist.• Not enough relevant data (e.g. exclusion by study type, single search, too tight inclusion criteria, not looked for documents where the same mechanism may be in operation).• Did a thematic analysis / did not apply a realist logic of analysis or used some other logic of analysis.• Confused intervention strategy with mechanism.• Unconfigured C, M, O (‘CMO soup’).Quality in reporting realist reviews• RAMESES I Project (2011-2013).• Reporting standards published:http://bmcmedicine.biomedcentral.com/articles/10.1186/1741-7015-11-21• 19-item checklist.• Has built in flexibility – acknowledging the variety in purpose and users’ needs of realist reviews.• Not all items need to be reported – but if an item is left out this should be highlighted and justified.Quality in execution – realist reviews• Quality standards available for peer-reviewers and also for funders.• 8 criteria and uses a rubric approach –inadequate/adequate/good/excellent.• Available on RAMESES Project website (www.ramesesproject.org) under Standards and Training materials.Realist review example – unpacking the problem• Review of interventions to improve the antimicrobial practice of doctors-in-training• Most interventions took a narrow focus on providing education• Outcomes were mixed or unclear• Unpacking revealed the stronger influence of hierachies in driving prescribing behaviour– Wanting to fit in– Following seniors– Reputation management• Papoutsi C, Mattick K, Pearson M, Brennan N, Briscoe S, Wong G. Social and professional influences on antimicrobial prescribing for doctors-in-training: a realist review. Journal of Antimicrobial Chemotherapy 2017; dkx194. doi: 10.1093/jac/dkx194. Realist review example – developing interventions• Review to develop interventional strategies to improve access to primary care for socioeconomically disadvantaged older people in rural areas• Programme theory of patient journey• Each stage’s outcome has a CMO configuration• Ford J, Wong G, Jones A, Steel N. Access to primary care for socioeconomically disadvantaged older people in rural areas: a realist review. BMJ Open 2016; 6:e010652. Realist review example – developing interventionsRealist review example – transferability of mechanisms• Review to develop a programme theory to understand if a piece of public health legislation was likely to be successfully implemented• Programme theory tested in a ‘desktop’ exercise on legislation banning smoking in vehicles carrying children• In 2010, no evaluation of such legislation existed• For the enforcement part of the programme theory we had to extrapolate from car seat legislation• Wong G, Pawson R, Owen L. Policy guidance on threats to legislative interventions in public health: a realist synthesis. BMC Public Health 2011; 11(222) • Pawson R, Wong G, Owen L. Known Knowns, Known Unknowns, Unknown Unknowns: The Predicament of Evidence-Based Policy. American Journal of Evaluation 2011; 32(4):518-546   Table 1. Context-Mechanism-Outcome Configurations for Team Processes Program Theory # Forming: Establishing social ground rules, learning about each other’s scope and roles, creating a psychologically safe workplace 1 CMOC: When team members have opportunities to interact with each other (C), perspective-taking occurs (M), laying the foundation for social bonds and trusting relationships (O).  2 CMOC:  When teams have dedicated supports in place (C),  team members have increased opportunities for perspective-taking (M), resulting in more awareness of each other’s roles and scopes (O)  3 CMOC: When team members are aware, interested and concerned about others (C), they strive to demonstrate carefulness in their actions and words (M), resulting in greater workplace psychological safety (O)   Storming: Working out conflict, narrowing the us-them gap 4 CMOC: When team members lack carefulness for others (C), there is decreased perspective-taking (M), resulting in more conflict, less willingness to work collaboratively, and a greater us-them gap (O).    Norming: Developing deeper, affective trust and a collective team identity 5 CMOC: When team members have planned time and space for ongoing opportunities to interact with each other (C), because of perspective-taking (M), they begin developing affective trust in one another (O).   6 CMOC: When team members openly share information and ideas (C), because of affective trust (M), they develop a collective team identity (O).   Performing: Managing ambiguity, inviting change and innovation 7 CMOC: When team members use carefulness and perspective-taking (C), because of deep trust (M) they are comfortable with ambiguity and have the capacity to champion innovation (O).    Table 2. Evidence-Informed Strategies for Key Team Processes Mechanisms Mechanism Strategies Perspective-taking • Routinized, frequent interactions • Briefing, huddles, debriefing • Team meetings Carefulness Relational education • Mindfulness • Appreciative Inquiry • Reflected Best Self Trust • Formal, facilitated orientation • Creation of team ground rules • Shadow experiences  

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            data-media="{[{embed.selectedMedia}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
https://iiif.library.ubc.ca/presentation/dsp.67246.1-0390457/manifest

Comment

Related Items