UBC Faculty Research and Publications

Building theories of knowledge translation interventions: Use the entire menu of constructs Brehaut, Jamie C; Eva, Kevin W Nov 22, 2012

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata

Download

Media
52383-13012_2012_Article_542.pdf [ 204.25kB ]
Metadata
JSON: 52383-1.0221340.json
JSON-LD: 52383-1.0221340-ld.json
RDF/XML (Pretty): 52383-1.0221340-rdf.xml
RDF/JSON: 52383-1.0221340-rdf.json
Turtle: 52383-1.0221340-turtle.txt
N-Triples: 52383-1.0221340-rdf-ntriples.txt
Original Record: 52383-1.0221340-source.json
Full Text
52383-1.0221340-fulltext.txt
Citation
52383-1.0221340.ris

Full Text

DEBATE Open AccessBuilding theories of knowledge translationinterventions: Use the entire menu of constructsJamie C Brehaut1,2* and Kevin W Eva3AbstractBackground: In the ongoing effort to develop and advance the science of knowledge translation (KT), animportant question has emerged around how theory should inform the development of KT interventions.Discussion: Efforts to employ theory to better understand and improve KT interventions have until recently mostlyinvolved examining whether existing theories can be usefully applied to the KT context in question. In contrast tothis general theory application approach, we propose a ‘menu of constructs’ approach, where individual constructsfrom any number of theories may be used to construct a new theory. By considering the entire menu of availableconstructs, rather than limiting choice to the broader level of theories, we can leverage knowledge from theoriesthat would never on their own provide a complete picture of a KT intervention, but that nevertheless describecomponents or mechanisms relevant to it. We can also avoid being forced to adopt every construct from aparticular theory in a one-size-fits-all manner, and instead tailor theory application efforts to the specifics of thesituation. Using audit and feedback as an example KT intervention strategy, we describe a variety of constructs(two modes of reasoning, cognitive dissonance, feed forward, desirable difficulties and cognitive load, communitiesof practice, and adaptive expertise) from cognitive and educational psychology that make concrete suggestionsabout ways to improve this class of intervention.Summary: The ‘menu of constructs’ notion suggests an approach whereby a wider range of theoretical constructs,including constructs from cognitive theories with scope that makes the immediate application to the new contextchallenging, may be employed to facilitate development of more effective KT interventions.Keywords: Theory, Audit and feedback, Knowledge translation, Constructs, Domains, Cognitive psychology,EducationBackgroundHealthcare research costs over 100 billion dollars annu-ally in North America alone [1,2]. This considerable in-vestment yields important new knowledge that cansignificantly improve the health of patients and popula-tions, provided that the knowledge is implemented ap-propriately. Over the last 20 years or so, it has becomeincreasingly apparent that ‘appropriate implementation’is an extraordinarily complex and multifactorial problem[3]. The techniques people have used to implement newknowledge have most often lacked any substantivejustification, and have instead been based on past prac-tice and logistical constraints rather than any in-depthunderstanding of what is likely to work. Perhaps not sur-prisingly, systematic reviews of many commonly usedimplementation techniques [4-8] have shown their ef-fectiveness to be highly variable. While these realizationshave coalesced into the science of implementation, alsoreferred to as knowledge translation (KT), much remainsto be understood about why strategies aimed at improv-ing the use of new research knowledge to improvehealthcare have proven so inconsistent in effect.An important debate in this developing discipline per-tains to the use of theory to understand the techniquesand processes underlying KT. Some have argued thatKT interventions are too heterogeneous for anything tobe gained by trying to develop a generalizable theory[9,10]. More recently, however, the literature has largely* Correspondence: jbrehaut@ohri.ca1Ottawa Hospital Research Institute, Clinical Epidemiology Program, TheOttawa Hospital, General Campus, 501 Smyth Road, Centre for PracticeChanging Research, Box 201B, Ottawa, ON K1H 8L6, Canada2Department of Epidemiology and Community Medicine, University ofOttawa, Ottawa, ON K1H 8M5, CanadaFull list of author information is available at the end of the articleImplementationScience© 2012 Brehaut and Eva; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of theCreative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use,distribution, and reproduction in any medium, provided the original work is properly cited.Brehaut and Eva Implementation Science 2012, 7:114http://www.implementationscience.com/content/7/1/114subscribed to the arguments put forward by Ecclesand others [3,11] that the benefits to be derivedfrom theory in terms of generalizability of findings andstandardization of methodology outweigh any risks.Early efforts to understand KT techniques and pro-cesses have focused on identifying which existing socialand health psychological theories can most successfullybe applied in new KT contexts [12]. This approach cor-rectly assumes that the psychological principles under-lying one area of human endeavor often transfer toothers [13] and, as such, may provide a head start to-wards identifying causal mechanisms that determine theeffectiveness of the KT intervention in question [3].Using existing theory (i.e., ‘a coherent and non-contradictory set of statements, concepts, or ideas thatorganises, predicts, and explains phenomena, events, be-havior, etc.’ [3,14]) has the further benefits of a founda-tion of both empirical work performed in other contextsand methodological innovations establishing strategiesfor measuring relevant constructs (i.e., the abstractedconcepts or explanatory variables on which theory isbuilt) [3].The literature has now seen the application of a varietyof theories to various KT endeavors (see Godin [15] fora review). Based on this work, a number of lessons canbe identified. First, the number of theories potentiallyavailable for study is enormous [16,17]. Second, theorymay guide implementation in many ways [18], rangingfrom closely directing the development and implementa-tion of the intervention, to crudely being used as acrutch to justify an intervention post hoc [19]. Third,considering theory alone, without empirical observationto identify the influence of a specific context, can missimportant information relevant to improving KT [9,12].Fourth, theories can differ in the roles they are intendedto serve, with some clearly designed to provide explan-ation and prediction and others being intended primarilyto serve communicative roles, a fact that can complicateapplication to new contexts [20]. In part because ofthese challenges, the majority of theory application workhas focused on a very small number of theories [15] andyielded only modest progress on issues about how to im-prove KT interventions [21]. The best example of this isthe Theory of Planned Behavior (TPB) [21,22], whichhas been attractive to KT researchers because of itsbroad scope (i.e., it is explicitly intended to be applicableto all voluntary human behaviors) and its demonstratedapplicability to a wide range of behaviors. Unfortunately,the TPB has proven less useful for generating clearrecommendations for improving KT.In response to these issues, some have called for anincreased focus on theory construction in KT research,rather than the wholesale application of existing theoriesto contexts that may extend beyond their originalmandate. For example, Rycroft-Malone [12] argues thatdevelopment of context-specific ‘micro-theories’ wouldresult in a better understanding of the KT issues that arespecific to different stakeholders, disciplines, and set-tings. This might develop important localized informa-tion, but the inevitable risks of coarse application ofsuch an approach would include lack of generalizabilityof findings and the necessity for considerable methodo-logical groundwork for each new context.We propose that there is a middle ground that mightbe usefully explored. Construction (as opposed to appli-cation) of a theory need not involve de novo, start-from-scratch theory building. Instead, when seeking causalmechanisms that contribute to successful KT, one couldcombine theory-building activities with individual con-structs from any number of relevant theories. We willrefer to this as the ‘menu of constructs’ approach, andargue that researchers looking to explain an area of KTshould consider the entire menu, not just constructsassociated with a particular theory.We see the menu of constructs approach as attractivefor a number of reasons. It allows for inclusion of onlythose constructs that are relevant to the new context, ra-ther than requiring transport of the entire theory. Veryoften, it is the individual constructs, rather than the the-ory as a whole, that recommends the theory in the firstplace. For example, the notion that channel factors (i.e.,features of the environment that lead individuals to actin particular ways) will influence a clinician’s behaviormay be more readily applied to KT interventionsthan the entire social psychological theory of situation-ism (i.e., the notion that the situation in which one findsthemselves is the dominant factor in determining anindividual’s behavior [23]) in which channel factors areembedded. Theory newly constructed from a menu ofconstructs can: leverage measurement/methodologicaladvances from the domain in which the various con-structs were generated; incorporate both theory-basedconstructs and components of the specific context andbehavior [24]; and propose KT interventions that arerooted in, but not restricted by, the larger body of theor-etical literature.The goal of this paper is to spur debate about therange of roles that theory (and theory-relevant con-structs) should play in the overall endeavor to improveKT interventions. Our central claim is that theory devel-opment may progress more quickly if we allow ourselvesto incorporate constructs derived from a range of theor-ies, rather than feeling restricted to align/justify/use anyparticular theory in its entirety, thus broadening and tai-loring the conceptual underpinning of specific KT inter-ventions. In the following sections, we make ourarguments in the context of audit and feedback (A&F)as an example KT intervention, but believe that theBrehaut and Eva Implementation Science 2012, 7:114 Page 2 of 10http://www.implementationscience.com/content/7/1/114general logic will apply more broadly. We begin byexplaining why we have chosen A&F as our example.We then provide some examples about how the menu ofconstructs approach is already being explored. We thendiscuss several examples of constructs that may informour understanding of KT interventions, but have notbeen considered in the context of a theory of A&F. Wediscuss how the menu of constructs approach relates toother emerging paradigms for theory use in KT, particu-larly the theoretical ‘domains’ approach suggested by thework of Michie et al. [17]. And finally, we address somelimitations and areas for future work suggested by thisapproach to theory building.DiscussionWhy study audit and feedback?A&F is a convenient term for a heterogeneous group ofinterventions centered around providing feedback onexisting practice to healthcare providers. A&F interven-tions involve the development of a summary of some as-pect of clinical performance (audit) over a specificperiod of time, and subsequent provision of that sum-mary (feedback) to individual practitioners, teams, orhealthcare organisations. A&F has been shown to be ef-fective in a wide variety of clinical contexts, and is oneof the most commonly employed and evaluated KTinterventions. However, there is enormous variability; arecent Cochrane review of 140 A&F trials showed highlyvariable effectiveness, ranging from substantial positiveeffects to null and even negative effects [25]. Such vari-ability is at least in part due to lack of understanding ofthe causal mechanisms underlying A&F interventions.The extensive A&F literature has recently been thesubject of a variety of theory-guided systematic reviews,because more standard meta-analytic subgroup analyses(e.g., group size, number of interventions) rarely shedlight on causal mechanisms. The most recent Cochranereview reported that the effectiveness of these interven-tions depends in important ways on how the feedback ispresented, such as the source, frequency, delivery for-mat, and whether there is a specific target and actionplan [25]. Two theory-specific reviews have targetedwhether the effectiveness of A&F interventions is relatedto Feedback Intervention Theory [26,27] and ControlTheory [28,29]. A recent review of the explicit role oftheory use in A&F trials shows that relatively few trial-ists appear to have considered any theory during the de-velopment of their interventions [19].These reviews show immense variability between A&Fstudies in terms of target audience, intervention details,targeted practice change, and context of the interven-tions. Without knowledge of the relevant causalmechanisms, one cannot predict whether a successfulintervention will generalize, learn much from failedinterventions, or successfully optimize future interven-tions [3]. As an analogy, studies assessing the effective-ness of new drugs would rarely be successful withoutconsiderable foundational work explicating the under-lying biological mechanisms. Without similar founda-tional work, KT interventions such as A&F are likely tocontinue to be hit-and-miss propositions. In the nextsection, we argue that theory construction using a menuof constructs approach may have advantages over simpleapplication of existing theories.Applying theories versus constructing theories using amenu of constructsInitial work applying theory to better understand KTtechniques and processes has been mostly drawn fromtheories of behavior from health and social psychology.For example, the Theory of Planned Behavior (TPB)describes changes in behaviors (e.g., smoking cessation,changes in antibiotics prescribing habits) as being pri-marily determined by individuals’ intentions to engage inthe behavior. Intention, in turn, is primarily determinedby three factors: attitude towards the behavior, subjectivenorms (what important others think of the behavior),and perceived behavioral control (whether the personfeels that the behavior is under their control). TPB is in-fluential in discussions around the use of theory in KT,in part because it can be usefully applied in so manycontexts (across 16 different studies of provider beha-viors, these constructs correlate strongly to changes intarget behavior, on average accounting for 31% of thevariability [30]), in part because it is a relatively simpletheory to describe and explore [3], and in part becauseof its ubiquity; until recently, the vast majority oftheory-informed efforts to change health behaviors, par-ticular health provider behaviors, involved versions ofthe TPB [21,30].Targeting broad, generalizable theories like the TPB asan initial step towards understanding KT interventionshas a lot to recommend it. Such theories focus on real-world, observable behavior as the key construct to beexplained, rather than, for example, theories of humanmemory that are built dominantly on human perform-ance in experimental settings and may be more difficultto generalize to non-laboratory-based settings. Simpleconstructs such as those comprising the TPB can bereasonably understood without an extensive disciplin-ary background, important in any interdisciplinaryfield. While constructs like ‘transfer appropriate proces-sing’ (i.e., the notion that the match between how infor-mation is encoded in memory and how it is to beretrieved will influence the likelihood its being remem-bered) [31] may well be relevant to many KT interven-tions, in its entirety it is a complex concept that isunlikely to be readily unpacked by non-specialists. TheBrehaut and Eva Implementation Science 2012, 7:114 Page 3 of 10http://www.implementationscience.com/content/7/1/114TPB specifically also comes with established methodolo-gies for measuring the relevant constructs [32], an ex-tremely useful criterion for content experts who may benew to the application of theory in their area.Despite these advantages, detailed theoretical under-standing of KT interventions requires investigation be-yond broad theories like TPB. For example, the TPB hasbeen criticized as a theory of KT intervention for beingbetter at explaining intention to engage in the behavior(on average, 59% of intention is explained) than it is atexplaining the behavior itself (31%) [30], for having rela-tively little to say about how to change and improve KTinterventions that have been found to be ineffective [21],and for focusing only on voluntary human behavior,when so much of health practice and behavior has atleast some automatic, rather than explicitly intentional,component [3].To us, these general criticisms suggest a need to ‘drilldown’ into specifics, to understand and describe moredetailed constructs underlying the contexts, interven-tions, and behaviors in question. Many such constructsexist within the discipline of cognitive psychology, thescientific discipline devoted to understanding the basicmechanisms underlying human thought, including per-ception, memory, categorization, and judgment and de-cision making [33]. Many cognitive constructs seem tohave face validity in the KT context and suggest specific,testable, predictions about how interventions might bemade more effective. As such, they should be exploredin order to examine their utility for describing and im-proving KT interventions.Some work has already begun to explore the practiceof combining constructs from different theories. Eccleset al. [34] conducted a postal survey of 230 Scottish gen-eral practitioners around management of upper respira-tory tract infections without antibiotics. Noting therange of health and social psychological theories avail-able and the lack of data on their relative merits, thestudy examined the extent to which constructs from arange of theories predicted hypothetical vignette-baseddecisions, and actual clinical behavior. Results showedthat the model that explained the most variance involvedconstructs from multiple theories, as opposed to modelsrestricted to an individual theory. A study looking at oralradiography behavior among 214 Scottish dental practi-tioners showed similar results (i.e., more varianceexplained when using constructs from multiple theoriesthan any individual theory on its own [35]). While Foy[36] provides a counterexample, these two studies pro-vide intriguing initial empirical evidence to support ourclaim that incorporation of constructs from differenttheories (i.e., what we are calling a menu of constructsapproach) may lead to advances in understanding KTinterventions.Examples of cognitive constructs worth exploringWe see A&F interventions as a series of mechanismsdesigned to improve alignment between a practitioner’spractice, the practitioner’s beliefs about his or her actualpractice, and best practices as defined by the broaderprofessional community. While all KT interventions seekto align actual practice with best practice, A&F is one ofthe few that also explicitly targets the fact that individualpractitioners rarely have ready access to accurate infor-mation on their practice patterns [37]. A great deal ofwork in cognitive and educational psychology may shedlight on the most effective mechanisms for enabling thisalignment, but remains wholly unexplored in the KT lit-erature. Because of the level of abstraction at whichthese theories were originally conceived, however, it isunlikely that any one theory will provide a complete pic-ture of how A&F may be optimized, thus creating theneed to pick and choose individual constructs from mul-tiple theories to determine how they might apply to spe-cific A&F contexts. Below, we present some examples ofconstructs that may suggest important causal mechan-isms related to A&F. All of these constructs have beenextensively studied, but few, if any, have been consideredin the context of KT interventions. We can, therefore,offer little empirical data as to their impact on the effect-iveness of A&F specifically. Instead our intent is simplyto indicate how identifying such constructs can make ex-plicit, testable predictions that can inform future re-search and development efforts in this applied domain.Two modes of reasoningOne of the most important theoretical perspectives tocome out of cognitive psychology is the notion of twomodes of reasoning [38,39], generically referred to asdual-processing theory. One mode, System 1 [38], canprocess information quickly, intuitively, and with rela-tively little effort. In medicine, development of know-ledge structures that allow complex decisions to bemade quickly is considered a cornerstone of medical ex-pertise [40,41] and likely accounts for a great many ofthe decisions made during the course of any health pro-vider’s day. The other mode, System 2, can be character-ized as slow, analytic, deliberative, and effortful. Patientcases where a provider must stop to think and problem-solve invoke System 2 processing.Some implications of this important dichotomy inhuman reasoning for KT have been outlined elsewhere[42,43]. A&F as a KT strategy most often invokes theSystem 2 mode of reasoning; information about currentpractice must be interpreted and understood, recom-mendations suggested by guidelines must be considered,and practice change implemented as deemed appropri-ate. The extent to which this effortful, deliberativeprocess can affect a practice decision that is governed byBrehaut and Eva Implementation Science 2012, 7:114 Page 4 of 10http://www.implementationscience.com/content/7/1/114System 1 processing is unclear, because the A&F litera-ture has not generally been informed by this theoreticalapproach; interventions are not designed with thesenotions in mind, and reports do not describe interven-tions in these terms. However, in the context of design-ing A&F interventions, dual-process theory can makeexplicit recommendations about how to improve inter-ventions including understanding the nature of theprocessing involved in the target decision, employingmulti-factor interventions to target different processingmodes, and considering cognitive strategies that take ad-vantage of the strengths of both forms of reasoning [44].Cognitive dissonance and information discountingDual processing suggests that a disconnect may exist be-tween the system targeted by A&F interventions and thesystem dominating actual decision-making, and that thismight help explain why simple provision of feedback viaan A&F task is not sufficient to ensure practice change.If this is true, one needs to ask what mechanisms mighthelp determine whether or not the feedback is effective.One lens we can apply to this problem is the notion ofcognitive dissonance [45,46], the distressing mental statethat arises when we find ourselves acting in a way that isinconsistent with something we believe (e.g., prescribingantibiotics, perhaps because one’s patients desire them,while believing their use should be limited) or holdingtwo conflicting beliefs at the same time (e.g., I am a goodphysician and my patients aren’t receiving best possiblecare). The state is sufficiently uncomfortable that manystudies have suggested we are highly motivated to re-duce the conflict experienced. The most common resultin these situations is that rather than altering behavioror abandoning a belief altogether, the tension is resolvedby the easier act of reinterpreting the beliefs (e.g., I am agood physician and my patients are different so theguidelines don’t apply).An important attribute of this information discountingstemming from cognitive dissonance is that it often hap-pens without conscious awareness (i.e., it resides in Sys-tem 1) [47]. There is a substantial literature [48] thatsuggests that these sorts of automatic reinterpretationsof available data are commonplace, preventing us fromknowing that we are falling prey to them. Indeed, thiskind of process has been proposed to be central to ageneralized psychological ‘immune system’ [49] we allpossess that involves automatic adaptive tendencies torationalize in a way that enables us to maintain a senseof well-being and personal strength. These constructsclarify why, instead of accepting feedback and adjustingbehavior accordingly, the result may more often be todiscount the feedback itself (e.g., ‘the data are not repre-sentative of my practice,’ ‘the data are biased,’ or ‘the datado not come from a credible source’). Failing to accountfor constructs such as these may prove a barrier to suc-cessful A&F intervention. Empowering feedback recipi-ents to determine what data will be most relevant totheir practice and how it is collected (i.e., guiding theaudit side of A&F) may reduce the likelihood that feed-back is discounted.Feedback and feed forwardWork on the Feedback Intervention Theory, alluded toearlier, suggests further ways in which theoretical con-structs can identify barriers that may threaten the effect-iveness of an A&F intervention. For example, FeedbackIntervention Theory might help explain why contextual-izing feedback through displays of how the target pro-vider compares to peers (a common strategy in A&F)can be precisely the wrong thing to do. Studies haveshown that drawing attention to the recipient’s self-efficacy (i.e., providing normative cues that prompt oneto think specifically about where one stands relative toothers along some continuum) can create a threat thatmakes it less likely that the feedback will have an influ-ence [26,50]. Rather than serving as a dispassionate indi-cator of where improvement is possible, such feedbackcan invoke the psychological immune system’s defensemechanisms, again yielding cognitive dissonance andleading to the data being discounted rather than alteringone’s self-assessments. Indeed, this can happen regard-less of the sign of the feedback; relatively poor perform-ance can encourage information discounting for the sakeof ego defense, while relatively good performance canlead one to believe any deviations from best practice areminor and, hence, there is little to be gained from con-tinued efforts at improvement.Presenting data in a manner that does not create con-flicting beliefs in the first place (i.e., minimizes cognitivedissonance by not engendering the concern that one’sperformance is substandard) may, therefore, provide animportant consideration for those designing A&F inter-ventions. Kluger and van Dijk’s [51] feed forward strat-egy offers an intriguing possibility that needs to betested in an A&F context. It involves interviewing theindividual about positive past experiences to help themestablish an internal standard of excellence and strength-ening memory traces of good performances that willinfluence later System 1 (similarity-based) processing,rather than focusing on the distance between the indivi-dual’s performance and an external standard. Earlyresults suggest the procedure has the potential to enableinsights into how performance can be improved withoutdamaging self-efficacy [51].Desirable difficulties and cognitive loadWhile concerns about the amount of time and cognitiveresources practitioners have available to dedicate toBrehaut and Eva Implementation Science 2012, 7:114 Page 5 of 10http://www.implementationscience.com/content/7/1/114contemplating practice improvement leads many inter-ventionists to design A&F interventions to be as simpleand accessible as possible, this approach does not considerthe clear gains to be derived from requiring the right kindof effort on the part of the target provider. Bjork has putforward the notion of desirable difficulties [52-54], whichsuggests that we are better able to learn, remember, andmake use of information when we are put in situationsthat induce errors. These models are aimed deliberately athelping people overcome the false perception that theyhave understood and learned material in a way that willenable its use in the future. For example, being tested onmaterial has robustly been shown to yield better learningthan studying the same material multiple times, eventhough study often yields feelings of fluency that we erro-neously infer to indicate that learning has taken place[55]. Testing (i.e., being required to effortfully retrieve in-formation from memory), increases the likelihood that wewill be able to retrieve the information from memoryagain in the future and makes it more likely that our at-tention will be productively focused on areas of knowledgethat require further study [55]. If implemented well, suchdesirable difficulties might also improve cognitive disson-ance by making it more difficult to discount informationthat one has exerted effort to collect.One can further specify this issue in terms of cognitiveload, a notion that requires us to consider the varioustypes of load that can impact on our mental processingcapacity. Intrinsic load (the amount of information to belearned), extraneous load (created by the way in which theinformation is presented), and germane load (theresources required of working memory to deal with intrin-sic and extraneous load) are believed to play different rolesin learning. Too many competing factors (e.g., having toread through pages of text in a busy clinical environmentto understand the feedback provided by an audit) createtoo much extraneous load and suboptimal learning. Pre-senting material with a degree of difficulty greater thanthe learner is prepared to process can similarly increaseintrinsic load to the point of disadvantage [56]. By con-trast, too little germane load (i.e., not engaging workingmemory to a great enough extent) can create a situationwhere feedback recipients passively accept information,but are not convinced sufficiently (or prompted to elabor-ate on the information enough) for the learning to have along-term impact. Considerable research, largely focusedon designing multimedia learning platforms, has demon-strated principles that can optimize these various kinds ofload [57-59]. Consideration of these principles in light ofA&F interventions may be a fruitful area of research.Communities of practice and adaptive expertiseWhile the examples used to this point have largely fo-cused on the psychology of the individual learner, A&Finterventions are often targeted at teams or practices ra-ther than individuals [60-65]. The notion of Communi-ties of Practice [66] describes that social networks ofindividuals who share a concern and interact regularlyaround that topic offer substantial benefits for learning.The three crucial characteristics, according to Wenger,are a clearly identifiable domain (i.e., an area of sharedcompetence that distinguishes members of the commu-nity from others), the community itself (with relation-ships nurtured to support and help one another in thegroup’s joint activity), and practice (i.e., activity orientedaround a shared repertoire of experience, expertise, andresources). Practicing in isolation has been found to be amain predictor of underperformance [67]. Such Com-munities of Practice might be thought of as informal,ongoing A&F opportunities, and suggest ways in whichrelationship-centered education [47] and mentoring canbe incorporated with opportunities for desirable difficul-ties and reduced cognitive dissonance to develop novelmodels of A&F.These models remind us of theoretical notions ofadaptive expertise [68,69], which suggest that expertiseshould not be conceived of as a ‘thing’ that can beachieved, but rather, should be considered an approachto continuous efforts at quality improvement. Engagingin this way requires a reward structure that the respect,nurturing, and collegiality of one’s peers can create. Indi-viduals need to feel safe in discussing their clinical prac-tice, but at the same time, simply conveying informationto them with no opportunity to discuss the issues withothers and come to some mutual understanding of howto alter performance appropriately increases the likeli-hood that one may inappropriately discount externaldata that should not be discounted. At the end of theday, these models, when combined with various politicaland economic theories, suggest that establishing a sys-tem whereby the reward structure for healthcare provi-ders encourages deliberate effort to engage with A&Fmay prove particularly influential in ensuring theireffectiveness.SummaryOur examples of theoretical constructs derived fromcognitive and educational psychology all have one thingin common. They all stem from theories that, because ofthe context and/or level of abstraction at which theywere originally developed, could not hope to provideanything like a full picture of an A&F intervention. As aresult, these theories would generally not be consideredrelevant to KT interventions, because the context forwhich they were developed is so far removed from com-plex A&F interventions. Nevertheless, they do providespecific, testable hypotheses about ways in which A&Finterventions might be improved. Thus the need forBrehaut and Eva Implementation Science 2012, 7:114 Page 6 of 10http://www.implementationscience.com/content/7/1/114adopting a menu of constructs approach; each theoret-ical construct mentioned (and the many more that werenot mentioned) promise productive lines of inquiry thatcan yield greater guidance regarding how to adapt A&Fstrategies to particular settings and how to productivelyand efficiently test the effectiveness of such strategies.Menu of constructs and the theoreticaldomains frameworkThis menu of constructs approach is not inconsistentwith an important new approach to the use of theory inKT research started by Michie et al. [17]. Noting thenumber of health and social psychological theories po-tentially relevant to such research, as well as the consid-erable overlap in constructs among them, this groupengaged in a consensus process that distilled from 33different theories a set of 12 behavior change ‘domains’agreed to be relevant to implementation research. Thesedomains are intended to be distillations of different, butrelated, constructs from different theories, ones thatnevertheless have common implications for behaviorchange and implementation research. This offers an ex-ample of how the menu of constructs approach might beimplemented. By beginning from these 12 domains (nowupdated to 13) [70], this Theoretical Domains Frame-work (TDF) offers a systematic means to consider a widerange of theoretical approaches, and to narrow one’ssearch for theories relevant to a specific KT context.The TDF has since been extended in at least two differ-ent ways. First, it has been directly employed to recom-mend specific implementation intervention techniques,based on expert agreement on what intervention techni-ques are implied by each domain [71,72]. Such an ap-proach can provide concrete guidance for interventionists,but may do so without explicit reference to specific theor-ies or constructs.A second approach, referred to as TheoreticalDomains Interviewing (TDI) [73,74], involves developingan intervention by interviewing or surveying the targetaudience and using the domains to prompt participantsto identify barriers and facilitators to the target behavior.Responses are then categorized into the most relevanttheory-specific constructs, and an intervention is devel-oped based on the recommendations of the theory withthe most constructs identified as being relevant to thetarget behavior. TDI therefore provides a systematic ap-proach to identifying which theory may be most appro-priate to the new context.This second approach to applying the TDF specificallyseeks to select ‘the most appropriate theories to developinterventions for changing specific behaviors’ [73]. Be-cause the goal is to identify what is relevant at the levelof the theory rather than the level of the construct, how-ever, the approach might fail to target constructs such asthose discussed in this paper. Because theories from cog-nitive psychology often seek to explain mechanisms ra-ther than behaviors (i.e., they explain at a lower level ofabstraction), they may not, in isolation, lend themselvesto explaining complex behaviors and, as such, exclusiveuse of TDI may lead us back to the issues that began thispaper (i.e., those that arise when individual theories areadopted in an all-or-none manner). This is likely the rea-son why the original TDF [17] combined much of cogni-tive psychology into a single construct (memory,attention, and decision processes); because of the levelof abstraction problem, it is not clear how the many the-ories within cognitive psychology might be relevant toimplementation. It is only at the level of the constructthat it becomes clear how such processes can inform KTinterventions.We propose a simple fix to marry the TDI with themenu of constructs approach that will simultaneouslyindicate how the menu of constructs approach might beproductively implemented. Rather than identifying con-structs in the interviews as a means to identifying themost relevant theory, in some cases it might be worth-while to consider all constructs deemed relevant in theinterviews, and use them to construct a new theory spe-cific to that KT context. This approach would allow de-velopment of interventions and theory that incorporatesconstructs from various theories. Such an approachwould be consistent with our menu of constructs idea,allow for the incorporation of constructs at differentlevels of abstraction, and also make use of the importantmethodological advances from the TDF and TDI.We believe this approach would help overcome somethinkers’ objections to the utility of using theory in KTresearch [9,10]. Use of theory need not be in oppositionto detailed empirical understanding, but instead shouldserve as an orienting conceptual framework that can beused iteratively to both guide and to be influenced byempirical observation. Much of the negative connotationthat can be associated with the word ‘theory’ comes fromconfounding of the term with high level conceptualiza-tions that may describe a problem well, but offer little inthe way of concrete guidance regarding specific mechan-isms whereby practice can be changed. Theory that is sobroad as to be applicable to any situation may inevitablybe so weak as to yield little more than adages that framean outcome after it has occurred. For example, theoriesthat conceive of creativity as a tendency to ‘think outsidethe box’ can provide adequate descriptions of activitiesthat define creativity, but ultimately offer little guidanceregarding how to do so effectively.The most effective methods for implementing thismenu of constructs approach have yet to be established.We plan to present experts in a wide range of theoreticaldomains with example A&F interventions, eliciting theirBrehaut and Eva Implementation Science 2012, 7:114 Page 7 of 10http://www.implementationscience.com/content/7/1/114opinions about theories and constructs that they feelmake testable predictions about how to improve theinterventions. Once a laundry list of constructs isassembled, we anticipate conducting pilot evaluations inthe form of written vignettes, usability testing sessions,or small scale randomized controlled trials to assesswhich candidate constructs warrant more formal evalu-ation. This pilot testing process could also incorporatethe results of bottom-up analyses of the target behaviorand context, of the sort recommended by others [75].LimitationsThis menu of constructs approach may be seen to havemuch in common with Bandura’s pejorative term‘cafeteria-style research’ [76]. He argues that pickingconstructs from various theories and recombining themcan lead to needless proliferation of essentially identicalconstructs with different names. When one’s goal is inte-gration of multiple overlapping theories into one all-encompassing theory, such proliferation is clearly aproblem. However, we see the process of theory develop-ment in KT as being distant from such a grand unifyingtheory. In the context of A&F interventions (only one ofmany possible KT interventions), we are only beginningto understand what factors predict an effective interven-tion. What is becoming clear is that the broad socialcognitive constructs such as those offered by the TPBhave not offered a sufficiently detailed theoretical de-scription to help us to consistently design effective A&Finterventions, and that the door must be opened to the-ory from a broader range of disciplines to understandthese complex interventions [77]. When higher-ordermodels do not provide sufficient help, ‘drilling down’ tomore complex and context-specific aspects of behaviorseems only sensible.Another potential limitation of the menu of constructsapproach is that by incorporating individual constructsindependently of the models with which they weredeveloped, one may lose some of the power of the ori-ginal theory, and potentially some of the meaning of theconstruct itself. Theories are comprised not only of con-structs, but of the proposed relationships between con-structs as well. Porting constructs into new contexts,separate from these relationships, may have unexpectedimplications for the utility of the construct. For example,will the ‘perceived behavioral control’ construct from theTPB have the same explanatory value independently ofthe other TPB constructs? We believe that this problemis one of validation. No one is suggesting that all poten-tially relevant constructs will prove useful in every con-text. Rather, adoption of a menu of constructs approachis meant to offer specific prompts that necessitate thevalidation of each construct within the context of thenew theory being built. In the newly resulting theoreticalcontext, any individual construct may or may not addexplanatory value, and cannot therefore be includedsolely based on its utility in its original theory.Finally, while in general we see the flexibility inherentin the menu of constructs approach to be a way forwardin the KT literature, that flexibility may sometimes bemore of a hindrance than a help. One of the attractionsof the TDF approach is that it boils down many theoriesinto a few key domains, which may be seen as moretractable from the point of view of designing implemen-tation strategies. In contrast, the menu of constructs ap-proach widens the number of constructs to considereven further, by incorporating theories that to date havenot been considered in a KT context. We feel that differ-ent tools will suit different purposes, and that furtherwork attempting to use the menu of constructs approachwill allow us to fruitfully explore these issues in moredetail.As with any component of evidence-based medicine,theory should be applied judiciously rather than adoptedlock, stock, and barrel with no consideration of the indi-vidual idiosyncrasies created by different contexts. Fur-thermore, as KT researchers, the goal of theory buildingshould remain firmly on how to create more effectiveinterventions. If the menu of constructs approach allowsus to develop a better understanding of the range of the-ories from multiple disciplines available to us, and to en-gage in systematic study of the applicability of theirconstructs, we will be better positioned to make clearrecommendations about how to implement newly devel-oped knowledge more effectively.Competing interestsThe authors declare that they have no competing interests.Authors’ contributionsBoth authors contributed to all aspects of the manuscript. Both authors readand approved the final manuscript.AcknowledgementsThis work was funded by a grant from the Canadian Institutes for HealthResearch (KTE 111–413). Dr. Brehaut holds a CIHR New Investigator Award inthe area of KT. The authors thank Drs. Anne Sales, Susan Michie, Noah Ivers,Heather Colquhoun, and Jeremy Grimshaw for conversations and commentson earlier drafts of this manuscript, Drs. Marie Johnston, Benjamin Gardner,and Falco F. Sniehotta for thoughtful reviews and comments, and KellyCarroll for help with manuscript preparation.Author details1Ottawa Hospital Research Institute, Clinical Epidemiology Program, TheOttawa Hospital, General Campus, 501 Smyth Road, Centre for PracticeChanging Research, Box 201B, Ottawa, ON K1H 8L6, Canada. 2Department ofEpidemiology and Community Medicine, University of Ottawa, Ottawa, ONK1H 8M5, Canada. 3Centre for Health Education Scholarship, Department ofMedicine, University of British Columbia, Vancouver, BC V5Z 4E3, Canada.Received: 6 January 2012 Accepted: 20 November 2012Published: 22 November 2012Brehaut and Eva Implementation Science 2012, 7:114 Page 8 of 10http://www.implementationscience.com/content/7/1/114References1. Canadian Institutes of Health Research: Health research roadmap: creatinginnovative research for better health and health care.http://www.cihr-irsc.gc.ca/e/40490.html.2. Centers for Medicare and Medicaid Services: National health expendituredata. https://www.cms.gov/nationalhealthexpenddata/.3. Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing thebehavior of healthcare professionals: the use of theory in promoting theuptake of research findings. J Clin Epidemiol 2005, 58:107–112.4. Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, et al:Effectiveness and efficiency of guideline dissemination andimplementation strategies. Health Technol Assess 2004, 8:iii–72.5. Jamtvedt G, Young JM, Kristoffersen DT, O'Brien MA, Oxman AD: Audit andfeedback: effects on professional practice and health care outcomes.Cochrane Database Syst Rev 2006, 2:CD000259.6. Grimshaw J, Eccles MP, Greener J, MacLennan G, Ibbotson T, Kahan JP, et al:Is the involvement of opinion leaders in the implementation of researchfindings a feasible strategy? Implement Sci 2006, 1:3.7. Foy R, Eccles MP, Jamtvedt G, Young J, Grimshaw JM, Baker R: What do weknow about how to do audit and feedback? Pitfalls in applying evidencefrom a systematic review. BMC Health Serv Res 2005, 5:50.8. Oxman A, Thomson M, Davis D, Haynes R: No magic bullets: a systematicreview of 102 trials of interventions to improve professional practice.CMAJ 1995, 153:1423–1431.9. Oxman A, Fretheim A, Flottorp S: The OFF theory of research utilization.J Clin Epidemiol 2005, 58:117–120.10. Fretheim A, Flottorp S, Oxman A: It is a capital mistake to theorize beforeone has data: a response to Eccle's criticism of the OFF theory ofresearch utilization. J Clin Epidemiol 2005, 58:19–20.11. Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Response to ‘The OFFtheory of research utilization’. J Clin Epidemiol 2005,58:117–118.12. Rycroft-Malone J: Theory and knowledge translation: setting somecoordinates. Nurs Res 2007, 56:S78–S85.13. Weber EU, Johnson EJ: Mindful judgment and decision making. Annu RevPsychol 2009, 60:53–85.14. Bem S, Looren-de Jong H: Theoretical issues in psychology. 1997.15. Godin G, Kok G: The theory of planned behaviour: A review of itsapplications to health behaviours. Am J Health Promot 1996,11:87–98.16. Grol R, Wensing M, Eccles M: Improving patient care: the implementation ofchange in clinical practice. Toronto: Elsevier Limited; 2005.17. Michie S, Johnston M, Lawton R, Parker D, Walker A: Making psychologicaltheory useful for implementing evidence based practice: a consensusapproach. Qual Saf Health Care 2005, 14:26–33.18. Michie S, Prestwich A: Are interventions theory-based? Development of atheory coding scheme. Health Psychol 2010, 29:1–8.19. Colquhoun H, Brehaut JC, Eva KW, et al: A systematic review of the explicituse of theory and theoretical constructs in randomized controlled trials of auditand feedback. [Manuscript in preparation].20. Buchanan DR: Beyond positivism: humanistic perspectives on theory andresearch in health education. Health Educ Res 1998, 13:439–450.21. Conner M, Sparks P: Theory of planned behavior and health behavior,Predicting health behaviour research and practice with social cognitionmodels. 2nd. Maidenhead: Open University Press; 2005:170–222.22. Ajzen I: The theory of planned behaviour. Organ Behav Hum Decis Process1991, 50:179–211.23. Ross L, Nisbett RE: The person and the situation: perspectives of socialpsychology. New York: McGraw-Hill; 1991.24. French SD, Green SE, O'Connor DA, McKenzie JE, Francis JJ, Michie S, et al:Developing theory-informed behaviour change interventions toimplement evidence into practice: a systematic approach using theTheoretical Domains Framework. Implement Sci 2012, 7:38.25. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, etal: Audit and feedback: effects on professional practice and healthcareoutcomes. Cochrane Database Syst Rev 2012, 6:CD000259.26. Kluger AN, DeNisi A: The effects of feedback interventions onperformance: a historical review, a mtea-analysis, and a preliminaryfeedback intervention theory. Psychol Bull 1996, 119:254–284.27. Hysong SJ: Meta-analysis: audit and feedback features impacteffectiveness on care quality. Med Care 2009, 47:356–363.28. Gardner B, Whittington C, McAteer J, Eccles MP, Michie S: Using theory tosynthesise evidence from behaviour change interventions: the exampleof audit and feedback. Soc Sci Med 2010, 70:1618–1625.29. Carver C, Scheier MF: On the self-regulation of behaviour. New York:Cambridge University Press; 1998.30. Godin G, Belanger-Gravel A, Eccles M, Grimshaw J: Healthcareprofessionals' intentions and behaviors: A systematic review of studiesbased on social cognitive theories. Implement Sci 2008, 3:36.31. Metcalfe J, Kornell N, Son LK: A cognitive-science based programme toenhance study efficacy in a high and low risk setting. Eur J Cogn Psychol2007, 19:743–768.32. Francis J, Eccles M, Johnston M, Walker A, Grimshaw J, Foy R, et al:Constructing questionnaires based on the Theory of Planned Behaviour:manual for health services researchers. 2004.33. Anderson JR: Cognitive psychology and its implications. New York: Freeman &Company; 1995.34. Eccles M, Grimshaw J, Johnston M, Steen N, Pitts N, Thomas R, et al:Applying psychological theories to evidence-based clinical practice:Identifying factors predictive of managing upper respiratory tractinfections without antibiotics. Implement Sci 2007, 2:26.35. Bonetti D, Pitts NB, Eccles M, Grimshaw J, Johnston M, Steen N, et al:Applying psychological theory to evidence-based clinical practice:identifying factors predictive of taking intra-oral radiographs. Soc Sci Med2006, 63:1889–1899.36. Foy R, Bamford C, Francis JJ, Johnston M, Lecouturier J, Eccles M, et al:Which factors explain variation in intention to disclose a diagnosis ofdementia?a theory-based survey of mental health professionals.Implement Sci 2007, 2:31.37. Audet AM, Doty MM, Shamasdin J, Schoenbaum SC: Measure, learn, andimprove: physicians' involvement in quality improvement. Health Aff(Millwood ) 2005, 24:843–853.38. Kahneman D: A perspective on judgment and choice: mapping boundedrationality. Am Psychol 2003, 58:697–720.39. Sloman S: The empirical case for two systems of reasoning. Psychol Bull1996, 119:3–22.40. Schmidt H, Norman G, Boshuizen H: A cognitive perspective on medicalexpertise: theory and implications. Acad Med 1990, 65:611–621.41. Norman G, Brooks L, Cunnington J, Shali V, Marriott M, Regehr G: Expert-novice differences in the use of history and visual information frompatients. Acad Med 1996, 71:S62–S64.42. Sladek RM, Phillips PA, Bond MJ: Implementation science: a role forparallel dual processing models of reasoning? Implement Sci 2006, 1:1–8.43. Brehaut JC, Hamm R, Majumdar S, Papa F, Lott A, Lang E: Cognitive andsocial issues in emergency medicine knowledge translation: a researchagenda. Acad Emerg Med 2007, 14:984–990.44. Eva KW: What every teacher needs to know about clinical reasoning. MedEduc 2005, 39:98–106.45. Festinger L: A theory of cognitive dissonance. Stanford, CA: StanfordUniversity Press; 1957.46. Tavris C, Aronson E: Mistakes were made (but Not by Me): why we justifyfoolish beliefs, bad decisions and hurtful acts. Orlando, FL: Harcourt; 2007.47. Eva KW, Armson H, Holmboe E, Lockyer J, Loney E, Mann K, et al: Factorsinfluencing responsiveness to feedback: on the interplay between fear,confidence, and reasoning processes. Adv Health Sci Educ Theory Pract2012, 17:15–26.48. Bargh JA, Chartrand TL: The unbearable automaticity of being. Am Psychol1999, 54:462–479.49. Nisbett R, Wilson T: The halo effect: evidence for unconscious alterationof judgments. J Pers Soc Psychol 1977, 35:250–256.50. Shute VJ: Focus on formative feedback. Rev Educ Res 2008, 78:153–189.51. Kluger AN, Van Dijk D: Feedback, the various tasks of the doctor, and thefeedforward alternative. Med Educ 2010, 44:1166–1174.52. Bjork RA, Linn MC: The science of learning and the learning of science:introducing desirable difficulties. APS Observer 2006, 19:39.53. Eva KW: Diagnostic error in medical education: where wrongs can makerights. Adv Health Sci Educ Theory Pract 2009, 14(Suppl 1):71–81.54. Bjork RA: Assessing our own competence: heuristics and illusions, Attentionand performance XVII: cognitive regulation of performance. Interaction oftheory and application. Cambridge, MA: MIT Press; 1999:435–459.55. Larsen DP, Butler AC, Roediger HL III: Test-enhanced learning in medicaleducation. Med Educ 2008, 42:959–966.Brehaut and Eva Implementation Science 2012, 7:114 Page 9 of 10http://www.implementationscience.com/content/7/1/11456. Guadagnoli M, Morin MP, Dubrowski A: The application of the challengepoint framework in medical education. Med Educ 2012, 46:447–453.57. Mayer RE: Multi-media learning. New York: Cambridge University Press; 2001.58. Mayer RE: Applying the science of learning to medical education. MedEduc 2010, 44:543–549.59. Issa N, Schuller M, Santacaterina S, Shapiro M, Wang E, Mayer RE, et al:Applying multimedia design principles enhances learning in medicaleducation. Med Educ 2011, 45:818–826.60. Eccles M, Rousseau N, Higgins B, Thomas L: Evidence-based guideline onthe primary care management of asthma. Fam Pract 2001, 18:223–229.61. Hemminiki E, Teperi J, Tuominen K: Need for and influence or feedbackfrom the finnish birth register to data providers. Qual Assur Health Care1992, 4:133–139.62. Hendryx MS, Fieselmann JF, Bock MJ, Wakefield DS, Helms CM, Bentler SE:Outreach education to improve quality of rural ICU care results of arandomized trial. Am J Respir Crit Care Med 1998, 158:418–423.63. Hillman AL, Ripley K, Goldfarb N, Nuamah I, Weiner J, Lusk E: Physicianfinancial incentives and feedback: failure to increase cancer screening inMedicaid managed care. Am J Public Health 1998, 88:1699–1701.64. Kafuko JM, Zirabamuzaale C, Bagena D: Rational drug use in rural healthunits of Uganda:effect of national standard treatment guidelines on rationaldrug use. 1st international conference on improving use of medications. 1999.65. Kinsinger LS, Harris R, Qaqish B, Strecher V, Kaluzny A: Using an officesystem intervention to increase breast cancer screening. J Gen Intern Med1998, 13:507–514.66. Lave J, Wenger E: Situated learning: legitimate peripheral participation.Cambridge: Cambridge University Press; 1991.67. Norman GR, Davis DA, Lamb S, Hanna E, Caulford P, Kaigas T: Competencyassessment of primary care physicians as part of a peer review program.JAMA 1993, 270:1046–1051.68. Mylopoulos M, Regehr G: Putting the expert together again. Med Educ2011, 45:920–926.69. Mylopoulos M, Woods NN: Having our cake and eating it too: seeking thebest of both worlds in expertise research. Med Educ 2009, 43:406–413.70. Cane J, O'Connor D, Michie S: Validation of the theoretical domainsframework for use in behaviour change and implementation research.Implement Sci 2012, 7:37.71. Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From theory tointervention: mapping theoretically derived behavioral determinants tobehaviour change techniques. Appl Psychol 2008, 57:660–680.72. Abraham C, Michie S: A taxonomy of behavior change techniques used ininterventions. Health Psychol 2008, 27:379–387.73. Francis JJ, Stockton C, Eccles MP, Johnston M, Cuthbertson BH, GrimshawJM, et al: Evidence-based selection of theories for designing behaviourchange interventions: using methods based on theoretical constructdomains to understand clinicians' blood transfusion behaviour.Br J Health Psychol 2009, 14:625–646.74. Islam R, Tinmouth AT, Francis JJ, et al: A cross-country comparison ofintensive care physicians' beliefs about their transfusion behaviour: aqualitative study using the theoretical domains framework. Implement Sci2012, 7:93.75. Michie S, van Stralen MM, West R: The behaviour change wheel: a newmethod for characterising and designing behaviour changeinterventions. Implement Sci 2011, 6:42.76. Bandura A: Health promotion from the perspective of social cognitivetheory. Psychol Health 1998, 13:623–649.77. Grol RP, Bosch MC, Hulscher ME, Eccles MP, Wensing M: Planning andstudying improvement in patient care: the use of theoreticalperspectives. Milbank Q 2007, 85:93–138.doi:10.1186/1748-5908-7-114Cite this article as: Brehaut and Eva: Building theories of knowledgetranslation interventions: Use the entire menu of constructs.Implementation Science 2012 7:114.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submitBrehaut and Eva Implementation Science 2012, 7:114 Page 10 of 10http://www.implementationscience.com/content/7/1/114

Cite

Citation Scheme:

        

Citations by CSL (citeproc-js)

Usage Statistics

Share

Embed

Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                        
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            src="{[{embed.src}]}"
                            data-item="{[{embed.item}]}"
                            data-collection="{[{embed.collection}]}"
                            data-metadata="{[{embed.showMetadata}]}"
                            data-width="{[{embed.width}]}"
                            async >
                            </script>
                            </div>
                        
                    
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:
http://iiif.library.ubc.ca/presentation/dsp.52383.1-0221340/manifest

Comment

Related Items