UBC Faculty Research and Publications

Increasing health policy and systems research capacity in low- and middle-income countries: results from… English, Krista M; Pourbohloul, Babak Jul 28, 2017

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12961_2017_Article_229.pdf [ 6.93MB ]
JSON: 52383-1.0349134.json
JSON-LD: 52383-1.0349134-ld.json
RDF/XML (Pretty): 52383-1.0349134-rdf.xml
RDF/JSON: 52383-1.0349134-rdf.json
Turtle: 52383-1.0349134-turtle.txt
N-Triples: 52383-1.0349134-rdf-ntriples.txt
Original Record: 52383-1.0349134-source.json
Full Text

Full Text

RESEARCH Open AccessIncreasing health policy and systemsresearch capacity in low- and middle-income countries: results from abibliometric analysisKrista M. English1,2* and Babak Pourbohloul1,2AbstractBackground: For 20 years, substantial effort has been devoted to catalyse health policy and systems research(HPSR) to support vulnerable populations and resource-constrained regions through increased funding, institutionalcapacity-building and knowledge production; yet, participation from low- and middle-income countries (LMICs) isunderrepresented in HPSR knowledge production.Methods: A bibliometric analysis of HPSR literature was conducted using a high-level keyword search. Health policyand/or health systems literature with a topic relevant to LMICs and whose lead author’s affiliation is in an LMICwere included for analysis. The trends in knowledge production from 1990 to 2015 were examined to understandhow investment in HPSR benefits those it means to serve.Results: The total number of papers published in PubMed increases each year. HPSR publications representapproximately 10% of these publications, but this percentage is increasing at a greater rate than PubMedpublications overall and the discipline is holding this momentum. HPSR publications with topics relevant to LMICsand an LMIC-affiliated lead authors (specifically from low-income countries) are increasing at a greater rate than anyother category within the scope of this analysis.Conclusions: While the absolute number of publications remains low, lead authors from an LMIC have participatedexponentially in the life and biomedical sciences (PubMed) since the early 2000s. HPSR publications with a topicrelevant to LMICs and an LMIC lead author continue to increase at a greater rate than the life and biomedicalscience topics in general. This correlation is likely due to increased capacity for research within LMICs and thesupport for publications surrounding large HPSR initiatives. These findings provide strong evidence that continuedsupport is key to the longevity and enhancement of HPSR toward its mandate.Keywords: Health policy, Systems research, Low- and middle-income countries, Knowledge production,Capacity-building* Correspondence: krista.english@ubc.ca1Complexity Science Lab, School of Population & Public Health, University ofBritish Columbia, Vancouver, British Columbia, Canada2Institute of Resources, Environment and Sustainability, University of BritishColumbia, Vancouver, British Columbia, Canada© The Author(s). 2017 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.English and Pourbohloul Health Research Policy and Systems  (2017) 15:64 DOI 10.1186/s12961-017-0229-1BackgroundA broad, interdisciplinary and applied field bringing to-gether economics, sociology, anthropology, political science,public health and epidemiology, health policy and systemsresearch (HPSR) is centred around how “health systemsand policies shape and are shaped, both by each other andby the broader determinants of health” [1]. HPSR thus hasmuch to contribute to health systems strengthening and, inturn, to the improvement of health outcomes, with the fielddefined more in terms of the questions it addresses ratherthan being rigidly constrained by methodological or discip-linary constraints [1, 2].The field of HPSR has evolved significantly over thepast few decades. The success of four Global Symposiaon Health Systems Research, and the establishment ofthe membership society Health Systems Global reflectthe crystallisation and growth of an HPSR researchcommunity. On the other hand, the development of theWHO Strategy on HPSR, the recognition of HPSR inthe World Health Report 2013 and the establishment ofthe Health Policy Leadership Initiative bringing togetherpolicymakers to identify HPSR research priorities under-score a greatly increased level of interest in HPSRamong policy and decision-makers [1].This paper seeks to take stock of the evolution ofHPSR production over these past decades through abibliometric analysis of HPSR publications for the period1990–2015. Given the applied, context-sensitive natureof HPSR knowledge, it is important that HPSR producedis relevant to specific country settings, this is particularlytrue for low- and middle-income countries (LMICs),where health systems strengthening efforts have oftennot been adequately informed by locally relevant re-search. In light of this, we specifically examined HPSRproduced in relation to LMICs. In addition to being in-dicative of the growth and evolution of the field, trendsin research production also serve as a useful proxy forresearch capacity. Given the need to understand theevolution of HPSR research capacity in LMICs, weexamined the production of HPSR in LMICs. Finally, itis important to know the distribution of researchproduction by topical areas within HPSR. This has beenachieved through an examination of HPSR researchproduction in terms of the six building blocks (BBs) ofthe health system as put forth by WHO.This is a particularly opportune moment for analysingthe growth and evolution of HPSR. The importance ofhealth systems strengthening and appropriate researchto inform these efforts has been brought to the fore bycrises including the Ebola epidemic [1]. At a broaderlevel, the Sustainable Development Goals (SDGs) havebrought attention to the importance of working acrosssectors and taking an integrated view of development.HPSR is an applied field, drawing on several disciplines,and thus has much potential to contribute towards theachievement of the SDGs [1]. Finally, it is a little over20 years ago that the decision was taken to establish anentity dedicated to HPSR, which took shape in 1999 asthe Alliance for Health Policy and Systems Research, aninternational partnership housed within WHO that hashad a crucial role in catalysing this evolution and build-ing the field of HPSR [3].The paper is divided into three sections, namely themethods used to carry out the bibliometric analysis anda presentation of the findings, followed by a discussionand conclusion.MethodsFor the purpose of this study, HPSR is defined asresearch on the health system functions of regulation,organisation, financing and delivery of services, as wellas broader determinants (such as social and economicpolicies directly affecting the health system) [2]. Itfocuses primarily upon the more upstream aspects ofhealth, organisations, policies and programmes, but doesnot address clinical management of patients or basicscientific research [2, 4].Bibliometric analysisAt this milestone in HPSR history, evaluation and reflec-tion of the contribution is captured through a variety ofquantitative analyses. A bibliometric analysis is one typeof quantitative analysis used to examine the productionof academic literature over time. It is used to assess theimpact of a field, researcher(s), or a paper over time.These methods are intended to infer a correlationbetween impact and influence and participation andconnectedness in the published literature. Participationis measured by frequency of publication, while connect-edness is measured by co-authorship (English K, GhaffarA, Shroff Z, Pourbohloul B, Health Policy and SystemsResearch Collaboration Pathways: Lessons from a Net-work Science Analysis. In Review, submitted to HealthResearch Policy and Systems, 2017).DatabasesEleven databases among the University of BritishColumbia Library Catalogue demonstrated an intellec-tual contribution to the field of HPSR, under traditionaland emerging subject areas (Additional file 1). Sincemany of these databases are not traditionally related tohealth policy, a preliminary review was conducted ofeach database using high-level keywords to identify theproportion of potentially relevant papers.Web of Science is the largest database and has ap-proximately 50% more publications than PubMed [5, 6].PubMed is the next largest but has approximately twiceas many health policy publications as Web of Science.English and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 2 of 13PubMed is also a free search engine accessing primarilythe MEDLINE database of references and abstracts onlife and biomedical sciences, and biomedical topics. Inthe remaining databases, the percentage of relevant pub-lications may be quite high but the absolute number isvery low. Therefore, PubMed was selected as thedatabase to analyse HPSR.Search strategyThere are five components for the search strategy. Ahigh-level search strategy was defined, suitable for themulti-decade timescale of a bibliometric and networkanalysis. Given that keywords, terms and topics maytrend over time, the high-level keywords selected wereubiquitous and consistent.Defining HPSRTo ensure inclusivity of publications related to HPSR, ahigh-level keyword search strategy was applied. Thisstrategy assumes that publications related to HPSRwould, at the very least, have the words (health ANDpolicy) OR “health system(s)” somewhere within theentire text of the publication. Once these publicationswere identified, additional keywords could be includedto refine the definition.The syntax (health AND policy) implies both termsare required in a single paper for inclusion. Alternatively,if a paper had the specific term “health system*” eitherindependently or in combination with (health ANDpolicy), it was also eligible for inclusion. Applying theasterisk implies all potential variants extending from theexpression shall also be included, such as “healthsystems”.In the literature, disciplinary inclusion can be broadwhile exclusion is better defined. PubMed includes adefined set of filters to identify specific topics related toclinical queries and medical genetics [7]. The exclusioncriteria can be applied to the search strategy using theBoolean operator, “NOT” thereby removing the irrele-vant clinical literature [8].The species filter was applied to restrict the results tohuman studies [9].Relevance to LMICsGiven the context sensitivity of the findings of HPSR, itis important that efforts to strengthen health systems inLMICs be informed by research that is produced specificto particular contextual settings.This analysis thus identifies the collection of paperswith its main topic focused on an issue relevant to aLMIC (referred to in the figures as “LMIC Topic”). Thetitle and abstract sections, denoted by the tag “Title/Ab-stract [TIAB]”, are intended to most concisely describethe main focus and purpose of a paper. Therefore, thesepapers can be efficiently identified by limiting the searchto the list of 135 low-income countries (LICs), lowermiddle-income countries (Lower-MIC) and uppermiddle-income countries (Upper-MIC) and synonymsfor ‘developing country’ that appear in the title andabstract [10]. This strategy is used in combination withthe keyword search strategy.LMIC authorsThese analyses are designed to help our understandingof the extent to which LMICs participate meaningfullyin the HPSR that is meant to support decision-makingcapacity in their countries. Between 1998 and 2014, onlythe first author’s affiliation was included in PubMed[11]. Identifying lead authors from LMICs is one meansto determine participation; frequency of publication andconnectedness of their co-authorship networks overtime are used as the metrics.To identify authors from LMICs, a combination ofeach of the 135 LMICs was used with PubMed’sadvanced search field builder [Affiliation]. An “LMICAuthor” was defined as a first author whose institutionalaffiliation/address included an LMIC, this address wasused as a proxy for country of residence.List and classification of countriesFor the fiscal year 2016, the World Bank has identified135 LMICs and 80 high-income countries (HICs).LMICs generally refers to the three sub-classificationsthat represent LMICs, inclusive of all LICs, Lower-MICsand Upper-MICs [12]. We have altered the syntax to re-duce confusion between lower middle-income countriesand low- or middle-income countries, both of which aretypically abbreviated as LMICs. In addition to theindividual name of each country, the strategy includessynonyms for developing countries, [dev countr*](Additional file 1). The inclusion of these terms capturespapers that may refer to developing countries moregenerally as a main topic (in the title and or abstract)without listing the name of the country explicitly.Analysis over timeA publication date filter was used to restrict the studiesto each and all years inclusive of January 1, 1990, toDecember 31, 2015. The range of years is meant to spanbeyond the inception of the Alliance for Health Policyand Systems Research to identify a baseline.ResultsPubMed is comprised of more than 26 million papers;almost 16.7 million of which were published betweenJanuary 1, 1990, and December 31, 2015, and 10.5million remain for the same period, once the humanspecies filter is applied (Fig. 1). This latter groupEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 3 of 13represents the baseline for this analysis and is used toshow the general increase in publications for the speci-fied period.Each individual component was examined independ-ently to understand its contribution to any topic inPubMed over time before moving on to the cumulativeanalysis. The number of publications per individualcomponent is higher than the compound effect.Frequency of publication per independent componentAmong all publications in PubMed, those that focus onlyon HPSR (blue bars) have been increasing slowly andsteadily since 1990. Publications that focus only on anissue relevant to an LMIC (and may or may not alsofocus on HPSR) (teal bars) typically lag behind HPSRpublications (Fig. 2). In contrast to these modest pro-gressions, first authors from LMICs have participatedexponentially in the life and biomedical sciences(PubMed) since the early 2000s. In the early 2000s, firstauthors from LMICs published about twice as many pa-pers in PubMed than the number of publication that fo-cused on a topic relevant to LMICs, this rate has sincesteadily increased and in 2015 – it was four times asmany publications. Given that there is one lead authorper publication in PubMed, the increasing share amongLMIC lead authors since 2000, is greater than that ofPubMed overall.Since we know publications overall are increasing, wewould like to understand whether the pace of publica-tion among HPSR papers, with a topic relevant toLMICs and lead authorship from an LMIC is underper-forming, on par, or outpacing life and biomedicalsciences in general.Subsequent analysis includes the cumulative effect ofHPSR literature combined with LMIC Topic and LMICAuthor. It can be assumed that approximately 2/3 (n ~4400) of HPSR publications with a topic relevant toLMICs have lead authors affiliated with HICs. Some ana-lysis identifies the absolute number of publications,while others show normalised slopes. Certain featuresare more visually apparent when the data is normalisedto 1 in the most productive year per category. It allowsfor a fair comparison between data of different scales(different denominators).Fig. 2 Contribution of each independent component among all life and biomedical sciences in PubMed over timeFig. 1 Number of PublicationsEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 4 of 13All publications in PubMed (limited to human species)have increased from just over 260,000 per year in 1990to the maximum thus far of approximately 615,000 in2014. In figure format, subsequent categories would bealmost invisible given the scale that would be requiredfor the vertical axis. In most figures, 2015 generallyappears to have fewer publications but this is due to alag between the publication date in some journals andtheir appearance (publication date) in PubMed. Giventime, this year will continue the upward trend seen in allprevious years.Number of publicationsFigure 3 demonstrates the contribution, in terms ofabsolute number, of HPSR and its sub-categories overtime. All three categories have been increasing since1990. HPSR publications consistently comprise approxi-mately 10% of all publications in PubMed. These arefollowed by HPSR publications that have a main topicfocused on any LMIC and the further sub-set of theaforementioned, which also have a first author whosemain affiliation is in an LMIC. The last categoryprovides a relatively small contribution (0.004%–0.067%depending on the year) to the body of knowledge inPubMed overall.It is worth noting the increase in HPSR publications inthe mid-1990’s and again a significant ramp up aroundthe time of the First Global HSR Symposium in 2010.Normalised comparison of HPSR literature to all PubMed1990–2015The normalisation in Fig. 4 clarifies the significant andincreasing contribution of the category HPSR + LMICTopic + LMIC Author compared to Fig. 3, in recentyears. Specifically, in 1990–2015, the trend in HPSRproduction, as measured by the high-level keywords,increased at a greater rate than publications in PubMedoverall. HPSR publications with a topic relevant toLMICs also increased at a greater rate than the afore-mentioned. During this period, HPSR publications witha topic relevant to LMICs and a lead author from anLMIC is clearly an emerging area and is on pace withthe previous category. In the early 2000s, there was avisible boost in this latter category to where it began tooutpace all other categories in the last decade. This in-crease in HPSR is likely due, in part, to the fact that,around the same time, authors from LMICs had anincreased contribution to life and biomedical sciences ingeneral (Fig. 2). This overall increase may be the resultof increased capacity in LMICs as per the intention ofthe Millennium Development Goals and/or many otherdevelopment programmes, including the Alliance forHealth Policy and Systems Research.The best linear fit of each time series is identified by theslope of that line (i.e. the coefficient x in the equations)(Fig. 4). Comparatively speaking, the higher the coefficientof x in the equation, the steeper the angle of the slope andthe greater the increase in contribution to the disciplineover time. The slope of the linear fit is important as it cor-relates to the amount of HPSR knowledge production,participation and institutional capacity. The data impliesthat the increase in publication frequency in HPSR litera-ture from 1990 to 2015 outpaced life and biomedical sci-ences in general (PubMed), and that the discipline isholding its momentum. The slope of sub-ranges of yearsmay also be reviewed to understand different patterns ofchange over time (Fig. 5). For example, there have beensome very progressive contributions by LMICs in the lastdecade, where the slope would be much steeper than overthe full 26-year period.The slope of all HPSR papers with a topic focused onLMICs is slightly steeper than the HPSR papers ingeneral. This is an effect of the cumulative combination,meaning that the number of HPSR papers being pub-lished with a focus on LMICs is increasing at a greaterrate than HPSR papers in general. The steepest slope isFig. 3 Absolute number of health policy and systems research publications in PubMed from 1990 to 2015English and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 5 of 13among HPSR papers with a topic focused on LMICs andwith a first author affiliated in an LMIC. While the abso-lute number remains low, the percentage of contributionis increasing at a slightly greater pace than the othercategories over time.While the largest increase in production of HPSR know-ledge is among LMIC first authors writing about LMIC-relevant topics (grey bars), we acknowledge that correlationdoes not imply causation. Nonetheless, this outcome maybe indicative of the positive effect of the ongoing effort toensure increased funding and institutional capacity-building,and that knowledge production continues to support vulner-able populations and resource-constrained settings. Theseresults provide strong evidence to demonstrate that contin-ued investment and evaluation will ensure success andmeaningful inclusion of the regions that HPSR supports.Comparison of productivity per 5-year intervalDuring the early 1990s, publication in all PubMed (blue)and HPSR + LMIC Topic + LMIC Author (grey) followeda similar, and almost parallel, trajectory (Fig. 5). From 1996to 2006, knowledge production among all categories wasquite irregular, with no clear pattern. After 2006, all sub-categories began increasing and were highly productive.Immediately following the inaugural HSR Global Sym-posium in 2010, there was a substantial increase in allcategories of HPSR knowledge production. HPSR +LMIC Topic + LMIC Author (grey), is on trend to sur-pass the continuing increases in all other categories.The most intriguing changes have occurred in the mostrecent years, whereby subsequent categories have begunto outpace the All PubMed (blue) group. The HPSR +LMIC Topic (green) and HPSR + LMIC Topic + LMICAuthor (grey) have improved so significantly that AllPubMed (blue) and HPSR (teal) appear to be declining inprominence relative to the former.Distribution and influence of BBs over timeHPSR focuses primarily upon policies, organisations andprogrammes, but does not address clinical managementof patients or basic scientific research (for example, oncell or molecular structures). Heath systems havehistorically been viewed within a framework of six BBs,namely Health Financing, Health Workforce, Informa-tion and Evidence/Research, Leadership and Govern-ance, Medical Products and Technologies and ServiceDelivery.In the literature, disciplinary inclusion within a BB canbe broad while exclusion is better defined, yet, in prac-tice, the boundaries are often blurred (Additional file 1).An approach to understanding the field of HPSR is toanalyse each of these components. At any given time,each of the six BBs is in different states of developmentand definition. Given this reality, the definition of someBBs is more challenging than for others. Additionally,attempting to compartmentalise the complexity of healthsystems into six discrete components may be an over-simplification.Fig. 4 Normalised comparison of each incremental category per yearFig. 5 Normalised comparison of the slopes per 5-year intervalsEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 6 of 13Below, HPSR publications that focus on an LMICTopic and any of the six BBs were examined over time.Given the blurred boundaries between the BB, a high-level definition was used, similar to the HPSR searchstrategy, to ensure inclusivity. The strategy includedHPSR + LMIC Topic + BB. As in the previous case, sub-sequent keywords could be added to further refine eachBB but may reduce the absolute number of publicationseligible for inclusion to below a reasonable threshold formeaningful analysis.HPSR publications with an LMIC topic by BBTopics related to the BBs (Fig. 6) referred to as Informa-tion and Evidence/Research represent almost 60% of theHPSR literature captured in the graph above. Researchand evidence is a prominent and resilient tenant ofHPSR. Its increasing prominence since the inception ofHPSR demonstrates its importance to the discipline.While attempts have been made to define HPSR by BB[13, 14, 15], the name of each BB has changed over theyears and therefore carries a different meaning/contextdepending on the publication (Additional file 1) [4, 14,16, 17]. There are inherent overlaps that make it impos-sible to disentangle the six BBs. In addition, there is noapparent benefit to the discipline in attempting to do so.Participation by country and income groupFigure 7 demonstrates the contribution to HPSR bycountry and income group. Each circle represents acountry and the colour represents the income group.The x-axis identifies the number of publications on anytopic in PubMed between 1990 and 2015. The y-axisidentifies the number of HPSR publications for the sameperiod. In each of the three graphs, the scale isdramatically different, this becomes more apparent inthe large combined graph (Fig. 8). The size of the circleis the percentage of HPSR publications of all PubMedpublications as per the lead author’s national affiliation.Generally, the number of publications with lead authorsfrom LMICs is fairly low, with the exception of a fewoutliers.Larger circles have a higher percentage of HPSRpublications. The large circles appearing closer to theorigin (O) indicates that, while the absolute number ofpublications might be low, there is still a higher percent-age of HPSR publications.Percentage of all HPSR/PubMed per LMIC groupThe main outliers among LICs (yellow circles) includeUganda, Tanzania, Ethiopia and Nepal (Fig. 7). LIC con-tributions appear to be the most horizontally diffused inthe Cartesian plane. India outperforms the next closestLower-MIC (blue circles), with 7-fold Egypt’s number ofpublications in all PubMed and greater than 3-fold thenumber of HPSR publications by Kenya. Among Upper-MIC (green circles), China publishes most frequently inboth categories but the proportion of HPSR is low rela-tive to the overall contribution to PubMed. South Africa,Mexico, Brazil, Iran and Turkey are also outliers with astronger contribution to HPSR literature (as per themore vertical spread along the y-axis).The perception of contribution may change oncethe graphs are combined and scaled. Relative to otherincome groups, the concentration of LICs is now dif-fused vertically. While the absolute numbers arelower, LICs contribute a significant proportion oftheir publications to HPSR (bigger circles distributedvertically along the y-axis).Fig. 6 Health policy and systems research publications that focus on a low- and middle-income country topic sub-divided by building block over timeEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 7 of 13Percentage of HPSR/PubMed per income groupFrequency of individual LMIC author publication andcitationAmong the approximately 7000 HPSR publications witha topic relevant to LMIC (HPSR + LMIC Topic), therewere 15,701 individual authors, 6940 of which are fromLMICs (Fig. 8). Of these, 82 had 10 or more publica-tions, 59% of which were affiliated in an LMIC. Whileonly six authors had 20 or more publications, four ofthese authors were from LMICs.A total of 118 authors have had publications cited 100times or more, 36% of these are affiliated in an LMIC;19 authors have been cited 200 times or more and allwere from Upper-MIC or HICs (58%); four authors havebeen cited more than 300 times, 75% were from HICs.The figures below feature each of the LMIC countryincome groups and the normalised number of publica-tions by lead authors from LMICs. The normalisedcomparison is advantageous over the frequency as coun-tries with many publications make those with fewer ap-pear visually insignificant. While there are 135 LMICs intotal, the graphs include approximately half this number.If there were fewer than 500 publications per countryfrom 1990 to 2015, an annual breakdown was not avail-able. Therefore, this is a compilation of all countrieswithin each income group that produced a total of morethan 500 publications over the full duration of the study.LICsFrequency of publication by LIC author affiliationFigure 9 demonstrates the absolute number of HPSR +LMIC Topic publications by lead authors from LICs.The year 2002 was a turning point for publication fre-quency, both in terms of absolute number and normal-isation. All countries begin to elevate from the x-axis atthis time. In the normalised figures, the thick blue lineFig. 7 Comparison of contribution to health policy and systems research versus all PubMed publications by authors from each country per low- andmiddle-income country income groupFig. 8 Relative comparison of contribution to health policy and systems research versus all PubMed publications by authors from each countryper low- and middle-income country income groupEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 8 of 13represents the baseline of all PubMed publications. Priorto 2002, authors from Zimbabwe published most fre-quently among all LICs and after this point their publi-cations dropped to among the lowest. At the same time,authors from Nepal, Uganda, Tanzania and Ethiopia in-creased their publication frequency to lead among LICs.By 2013, Malawi, Burkina Faso, Zimbabwe, Guinea,Cambodia, Mozambique and Mali saw a slight increasein publication by authors from their countries. Gambia,Togo and Madagascar saw little to no increase in publi-cation frequency for the duration of the study period.Normalised comparison of LIC authored publicationsFigure 10 shows a normalised comparison of LICauthored publications. Frequency figures are notdepicted in subsequent income groups. The dispropor-tionately large contribution of India (Lower-MIC) andChina (Upper-MIC) appear to diminish the contributionof all other countries rendering them practicallyindistinguishable.Lower-MICsAuthors from India produced almost 10-fold that of theauthors from the next most frequently published Lower-MIC. Nigeria and Egypt were a distant second and third,respectively, until 2009, at which point they switchplaces. As in LICs, there appeared to be a shift around2002, whereby Pakistan separated from the other coun-tries to reach the fourth place. Kenya and Bangladeshemerged to follow by the mid-2000s.Normalised comparison of Lower-MIC authoredpublicationsIn the normalised Fig. 11 above, countries begin toelevate from the x-axis in the late 1990s. Papua NewGuinea is clearly outpacing PubMed and other Lower-MICs until the late 1990s. Authors from Côte d'Ivoire,Senegal, Ukraine and Nigeria lead Lower-MIC publica-tions around the turn of the century. Georgia, Pakistanand India show improved proportionality after the mid-2000s, keeping pace with the aforementioned countries.Upper-MICsSimilar to India among Lower-MICs, authors fromChina significantly out-produced all other authors fromUpper-MICs across the study period, with the trend be-ing overt in 2000. In 1990, Mexico was a distant secondbehind China. Authors from Mexico published consist-ently, with a slight increase over all years and holdingthe fifth place among all Upper-MIC. Around the year2000, authors from Turkey and Brazil contributed to adistant second and third place, respectively, beforeswitching in 2010, when Brazilian authors publish morefrequently than those from Turkey. In terms of fre-quency of publication, the top countries were consistentacross all years.Normalised comparison of Upper-MIC authoredpublicationsAuthors from Venezuela, Jamaica, Cuba and Bulgaria alltypically outpaced PubMed overall (Fig. 12). This charac-teristic was generally less common among individualcountries within the LICs and Lower-MICs.Evolution of HPSR themesThere was a distinct difference between the MedicalSubject Headings (MeSH) terms assigned and keywordsselected for a paper by the authors.The National Library of Medicine MeSH are a con-trolled vocabulary of biomedical terms used to describethe subject of each journal article in MEDLINE. Skilledsubject analysts examine journal articles and assign theFig. 9 Frequency of publication by low-income country author affiliationEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 9 of 13most applicable MeSH terms – typically 10–12. Apply-ing the MeSH vocabulary ensures that articles are uni-formly indexed by subject, regardless of the author’ssuggested keywords [17]. Among HPSR publications,MeSH terms identify the species in every paper, the sexand age group of the population under study are alsogiven priority.The frequency of author-assigned keywords is signifi-cantly lower and less standardised, but arguably morerepresentative of the papers’ topic. Author-assignedkeywords among HPSR publications focus on the topicof the paper, with little emphasis on demographic infor-mation unless it pertains to the socioeconomic status asis relevant to LMICs.This analysis examines MeSH terms because the vastmajority (approximately 6/7 papers) do not have anyauthor-assigned keywords from 2001 to 2011. The rea-son for this is unclear, but it is possible that PubMed didnot require that bibliometric field during that period.We welcome future analysis to identify interestingfeatures about trends in HPSR topics published overtime. Please see Additional file 1 for a comparison ofterms available.The resulting terms and themes are significant to un-derstanding the discipline as they were identified fromthe high-level keyword search used to capture the HPSRliterature. The figures below demonstrate the dynamictrends in important MeSH themes over time.Top 10 MeSH terms per yearOverall, 75,704 MeSH terms were assigned to 7009HPSR papers with a topic relevant to LMICs. Using thetop 10 assigned terms for each publication year, we cannote the changing trends in MeSH assignment (Fig. 13).While approximately the first five terms were consist-ently assigned, subsequent terms demonstrate the shiftsin policy interest.Top 10 MeSH terms per year – excluding demographics andgeographyBy removing the MeSH terms limited to sex, age, demo-graphics and specific countries, the terms remainingprovide insight on the HPSR topics of interest over time.‘Health Policy’ and ‘Developing Countries’ are present inall years from 1990 to 2015 (Fig. 14). In the middle ofthe graph, from 1995 to 2008, ‘Health Care Reform’ wasFig. 10 Normalised comparison of publications by first authors from low-income countriesFig. 11 Normalised comparison of publications by first authors from lower-middle income countriesEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 10 of 13a prominent issue. ‘Health Services Accessibility’ hasprogressively increased in prominence since the late1990s. While, ‘Delivery of Health Care’ saw a slow startin the 1990s, but significantly increased in prominencesince 2000, to represent a major share of the HPSRMeSH topics since the mid-2000s.DiscussionThis analysis has primarily examined HPSR using ahigh-level keyword search strategy, allowing the trendsto organically emerge from countries, regions and MeSHterms over time. It is worth noting that, occasionally,MeSH terms are also added/removed based on new re-search, social relevance and/or political policies, this isspecifically evident given addition of the term ‘HealthCare Reform’ in 1994.Historically, analysis incorporated the six BBs, but thereare some noteworthy limitations to using the six BBs toanalyse HPSR. The current BBs are somewhat arbitraryand these themes have evolved since their initial introduc-tion. In addition, there are inherent overlaps that makethem impossible to disentangle and there is no apparentbenefit to the discipline in attempting to do so.Each of the individual BBs are in a very different state ofdefinition. The methods and resulting keywords of pub-lished bibliometric strategies were reviewed [13, 15, 18, 19].The results varied greatly and some studies only definedHPSR in terms of the BB, counterintuitively leaving out‘health policy’ publications entirely. While some BBs wereextensively defined in terms of quantity of keywords, otherswere very limited and apparently under- or miss-represented by the MeSH terms used (i.e. Information andEvidence/Research was described as ‘Information Systems’and defined using MeSH terms related to patient/drug re-cords). Few strategies cited the sources from which key-words were drawn, and therefore verification, rationale andFig. 12 Normalised comparison of publications by first authors from upper-middle income countriesFig. 13 Top 10 MeSH terms assigned to health policy and systems research + low- and middle-income country topic publications per yearEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 11 of 13limits of the inclusion criteria were difficult to determine.Clinical studies were not explicitly excluded, as per Pub-Med’s prescribed filters (Additional file 1) [20].It is difficult to assess whether LMIC authors withmultiple affiliations might prefer indicating either theirLMIC or HIC affiliation. This could misrepresent thedistribution of authors by country classification. To bet-ter assess this, we would require knowledge about howmany LMIC authors have dual affiliation (both LMICand HIC) and, among these, how many would opt to in-dicate their HIC affiliation when writing about and forLMICs. In addition, there is a possibility that LMICs areover-represented in unindexed journals due to languagebarriers and/or restrictions by National Library ofMedicine, etc. Regardless, a database is required, someare indexed and some are not. Unfortunately, there is noway to know what proportion this might represent,whether it is ubiquitous across all income regions orwhether it is significant.In general, bibliometric analysis examines the fre-quency of publication over time. Co-authorship andcitation analysis are an extension of this and are bestunderstood using network analysis. We did not includea list of most-frequently published authors so as to avoidsingling out individuals.ConclusionsTwo decades ago, participation in HPSR by LMICs waslow and level. The Alliance for Health Policy andSystems Research began its work in the early 2000s. Inlooking back via bibliometric analysis, there is a correl-ation between the timing of specific initiatives resultingfrom major meetings or reports and increases in thepublished literature on HPSR that follow. Around themillennium, the momentum from global initiativesfocused around HPSR had shifted the knowledgeproduction, interest in and participation by LMICs.As the Alliance celebrates 20 years of a milestone meet-ing on HPSR, the momentum for HPSR continues to grow.The progressive advocacy for meaningful participation con-tributed support for the exponential increase in LMIC au-thors publishing in the life and biomedical sciences. Thisincreased regional capacity continues to facilitate growth inthe literature published on topics relevant to LMICs, suchthat it has been increasingly outpacing PubMed as a whole(as seen in figures showing normalised slopes). To date, theincreased participation shows no sign of slowing down. Asthis evolution continues, synergies and collaboration leadto a new level of sustained capacity for the individuals,institutions and regions.The knowledge gained since the introduction of HPSRtwo decades ago solidified the necessity of using a com-plexity lens to study health systems as a complex prob-lem. There are many interacting factors that affect andinfluence each other in different ways over time. Furtheranalysis of the abovementioned synergies would benefitfrom network analysis.There is another great opportunity to refine the func-tional components used to define HPSR. MeSH termsand keyword trends demonstrate the evolution andemergence of relevant issues over time. These trendshelp clarify where and how to provide additional supportand will highlight regions that would benefit from syner-gies and capacity-building efforts. Understanding andFig. 14 Top MeSH terms assigned to health policy and systems research + low- and middle-income country topic publications, excludingdemographics and geography, per yearEnglish and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 12 of 13defining HPSR is the necessary foundation for robustanalysis of the discipline. Therefore, future work shouldinvolve strengthening our understanding, clarifying thescope and definitions for each of the themes (i.e. BBs)used to measure HPSR. A collaborative process to de-scriptively and methodologically define the inclusive andexclusive criteria would greatly benefit future analysis ofthe field.As we look forward toward achieving the SDGs, theoutcome of this analysis may be indicative of the positiveeffect of the ongoing effort to ensure increased funding,institutional capacity-building and knowledge produc-tion to continue to support vulnerable populations andresource-constrained settings.Additional fileAdditional file 1: Supplementary Material. (DOCX 111 kb)AbbreviationsBBs: Building blocks; HICs: High-income countries; HPSR: Lealth policy andsystems research; LICs: Low-income countries; LMICs: Low- or middle-income countries; Lower-MICs: Lower middle-income countries;MeSH: Medical subject headings; SDGs: Sustainable development goals;Upper-MICs: Upper middle-income countriesAcknowledgementsNot applicable.FundingThis work was supported by The Alliance for Health Policy and Systems Research.The funders had no role or influence in the design or conduct of the study.Availability of data and materialsThe datasets generated and/or analysed during the current study areavailable from PubMed, https://www.ncbi.nlm.nih.gov/pubmed/.Authors’ contributionsKE and BP contributed to the conception, methodological design, andcollection and analysis of data. Both authors contributed to theinterpretation of results and drafting of the manuscript. Both authors readand approved the final manuscript.Ethics approval and consent to participateNot applicable.Consent for publicationNot applicable.Competing interestsThe authors declare that they have no competing interests.Publisher’s NoteSpringer Nature remains neutral with regard to jurisdictional claims inpublished maps and institutional affiliations.Received: 30 January 2017 Accepted: 29 June 2017References1. World Health Organization. Alliance for Health Policy and Systems Research.Strategic Plan 2016–2020. Investing in Knowledge for Resilient HealthSystems. 2016. http://www.who.int/alliance-hpsr/resources/investing-knowledge/en/. Accessed 23 May 2017.2. World Health Organization. Alliance for Health Policy and Systems Research.What is Health Policy and Systems Research (HPSR)? http://www.who.int/alliance-hpsr/about/hpsr/en/. Accessed 8 Aug 2016.3. Swedish International Development Cooperation Agency and the RoyalMinistry of Foreign Affairs, Norway. Alliance for Health Policy and SystemsResearch. Report and proposals from an international consultation. 1997.http://www.who.int/rpc/meetings/REVIEW.pdf?ua=1. Accessed 13 Jul 2017.4. World Health Organization, Alliance for Health Policy and Systems Research.Bennett S. Briefing Note 1: What is Health Policy and Systems Research andWhy Does it Matter? 2007. http://www.who.int/alliance-hpsr/resources/Alliance%20HPSR%20-%20Briefing%20Note%201.pdf. Accessed 8 August 2016.5 Clarivate Analytics. 2107. http://wokinfo.com. Accessed 29 Mar 2017.6 National Center for Biotechnology Information, U.S. National Library of Medicine.2017. https://www.ncbi.nlm.nih.gov/pubmed/. Accessed 29 Mar 2017.7 National Center for Biotechnology Information, U.S. National Library ofMedicine. Clinical Queries Filters. PubMed Help 2015. http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.Clinical_Queries_Filters. Accessed 26Oct 2016.8 National Center for Biotechnology Information, U.S. National Library ofMedicine. Combining Search Terms with Boolean Operators (AND, OR,NOT). PubMed Help 2015. http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.Combining_search_terms_with_B. Accessed 26 Oct 2016.9 National Center for Biotechnology Information, U.S. National Library ofMedicine. Species. PubMed Help 2015. http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.Species. Accessed 10 Oct 2016.10 National Center for Biotechnology Information, U.S. National Library ofMedicine. Title/Abstract. PubMed Help 2015. http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.TitleAbstract_TIAB. Accessed 26 Oct 2016.11 National Center for Biotechnology Information, U.S. National Library ofMedicine. Affiliation (AD). https://www.nlm.nih.gov/bsd/mms/medlineelements.html#ad. Accessed 26 Mar 2017.12 The World Bank. Country and Lending Groups. http://data.worldbank.org/about/country-and-lending-groups. Accessed 26 Aug. 2015.13 Yao Q, Chen K, Yao L, et al. Scientometric trends and knowledge maps ofglobal health systems research. Health Res Policy and Syst. 2014;12:26.14 World Health Organization. Monitoring the Building Blocks of HealthSystems: A Handbook of Indicators and their Measurement Strategies. 2010.http://www.who.int/healthinfo/systems/WHO_MBHSS_2010_full_web.pdfAccessed 17 Jan 2017.15 Adam T, Ahmad S, Bigdeli M, Ghaffar A, Røttingen JA, Noor AM. Trends inhealth policy and systems research over the past decade: still too littlecapacity in low-income countries. PLoS ONE. 2011;6(11):e27263.16 Uneke CJ, Ezeoha AA, Ndukwe CD, Oyibo PG, Onwe F, Igbinedion EB,Chukwu PN. Operational Manual for Strengthening Institutional Capacity toEmploy Evidence in Health Policymaking for Developing Countries: TheNigeria Experience. Geneva: WHO; 2010. http://www.who.int/alliance-hpsr/projects/alliancehpsr_opmanualstrengtheninginstitutionalcapacitynigeria.pdf.Accessed 17 Jan 2017.17 National Center for Biotechnology Information, U.S. National Library ofMedicine. MeSH Terms. PubMed Help 2015. http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.MeSH_Terms_MH. Accessed 28 Jan 2016.18 Tricco A, Runnels V, Sampson M, Bouchard L. Shifts in the use of populationhealth, health promotion, and public health: A bibliometric analysis. Can JPublic Health. 2009;99(6):466–71.19 Deshazo JP, Lavallie DL, Wolf FM. Publication trends in the medicalinformatics literature: 20 years of “Medical Informatics” in MeSH. BMC MedInform Decis Mak. 2009;9:7. doi:10.1186/1472-6947-9-7.20 National Center for Biotechnology Information, U.S. National Library ofMedicine. Clinical Queries Filters. PubMed Help 2015. http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.Clinical_Queries_Filters. Accessed 26Aug 2016.English and Pourbohloul Health Research Policy and Systems  (2017) 15:64 Page 13 of 13


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items