@prefix vivo: . @prefix edm: . @prefix ns0: . @prefix dcterms: . @prefix skos: . vivo:departmentOrSchool "Applied Science, Faculty of"@en, "Non UBC"@en, "Nursing, School of"@en ; edm:dataProvider "DSpace"@en ; ns0:identifierCitation "BMC Medical Research Methodology. 2016 Dec 05;16(1):169"@en ; ns0:rightsCopyright "The Author(s)."@en ; dcterms:creator "Young, Honor"@en, "Költő, András"@en, "Reis, Marta"@en, "Saewyc, Elizabeth Marie"@en, "Moreau, Nathalie"@en, "Burke, Lorraine"@en, "Cosma, Alina"@en, "Windlin, Béat"@en, "Gabhainn, Saoirse N."@en, "Godeau, Emmanuelle"@en ; dcterms:issued "2017-01-21T03:49:48"@en, "2016-12-05"@en ; dcterms:description """Background: This paper describes the methodological developments of the sexual health items included in the Health Behaviour in School-aged Children (HBSC) study since their mandatory inclusion in the study in 2002. The current methodological, ethical and pedagogical challenges in measuring young people’s sexual health behaviours are discussed along with the issues associated with the sexual health items introduced to the HBSC study in 2002. The development and piloting of new cross-national items for use in the 2013/14 HBSC data collection are presented and discussed. Methods An international pilot study was undertaken to determine the impact of these proposed changes. Questionnaires and classroom discussion groups were conducted in five pilot countries in 2012/2013 (France, Hungary, Ireland, Portugal and Romania) with a total of 612 school-aged children (age M = 15.55 years, SD = 0.95). Results The majority of participants in each country provided positive feedback about the appropriateness of the questions. Some small cross-national differences were found in the self-reported quantitative data relating to the appropriateness of the questions (χ2 = 22.831, df = 9, p = .007, V = .117). Qualitative feedback suggests that for the vast majority of students the phrasing and age-targeting of the questions were considered appropriate. With the exception of a small number of respondents who commented on the clarity and/or personal nature of the content, no specific issues with the questions were identified. Conclusions These findings provide guidance on the answerability (including the extent of missing and inconsistent data), understandability, acceptability (including in different cultures) and relevance of questions to potential participants. The findings from the pilot study suggest that in general, the questions are understandable, acceptable, and of a high priority to the target population, and that the simplification has significantly reduced the proportion of missing data. The new developments thus enhance the capacity of the questions to measure cross-nationally, sensitive aspects of young people’s sexual behaviour. These questions were included in the 2013/2014 round of the HBSC survey and will continue to be used to monitor trends in adolescent sexual health and behaviours, and to inform and influence health services and health education policy and practice at local, national and international levels."""@en ; edm:aggregatedCHO "https://circle.library.ubc.ca/rest/handle/2429/59885?expand=metadata"@en ; skos:note "RESEARCH ARTICLE Open AccessSexual Health questions included in theHealth Behaviour in School-aged Children(HBSC) Study: an internationalmethodological pilot investigationHonor Young1* , András Költő2,3, Marta Reis4,5,6, Elizabeth M. Saewyc7, Nathalie Moreau8, Lorraine Burke9,Alina Cosma10, Béat Windlin11, Saoirse Nic Gabhainn9 and Emmanuelle Godeau12AbstractBackground: This paper describes the methodological developments of the sexual health items included in theHealth Behaviour in School-aged Children (HBSC) study since their mandatory inclusion in the study in 2002. Thecurrent methodological, ethical and pedagogical challenges in measuring young people’s sexual health behavioursare discussed along with the issues associated with the sexual health items introduced to the HBSC study in 2002.The development and piloting of new cross-national items for use in the 2013/14 HBSC data collection arepresented and discussed.Methods: An international pilot study was undertaken to determine the impact of these proposed changes.Questionnaires and classroom discussion groups were conducted in five pilot countries in 2012/2013 (France,Hungary, Ireland, Portugal and Romania) with a total of 612 school-aged children (age M = 15.55 years, SD = 0.95).Results: The majority of participants in each country provided positive feedback about the appropriateness of thequestions. Some small cross-national differences were found in the self-reported quantitative data relating to theappropriateness of the questions (χ2 = 22.831, df = 9, p = .007, V = .117). Qualitative feedback suggests that for thevast majority of students the phrasing and age-targeting of the questions were considered appropriate. With theexception of a small number of respondents who commented on the clarity and/or personal nature of the content,no specific issues with the questions were identified.Conclusions: These findings provide guidance on the answerability (including the extent of missing andinconsistent data), understandability, acceptability (including in different cultures) and relevance of questions topotential participants. The findings from the pilot study suggest that in general, the questions are understandable,acceptable, and of a high priority to the target population, and that the simplification has significantly reduced theproportion of missing data. The new developments thus enhance the capacity of the questions to measure cross-nationally, sensitive aspects of young people’s sexual behaviour. These questions were included in the 2013/2014round of the HBSC survey and will continue to be used to monitor trends in adolescent sexual health andbehaviours, and to inform and influence health services and health education policy and practice at local, nationaland international levels.Keywords: Adolescent sexual health, Adolescent sexual behaviour, Self-completion, Questionnaire designinconsistencies, Missing data* Correspondence: Youngh6@cardiff.ac.uk1Centre for the Development and Evaluation of Complex Interventions forPublic Health Improvement (DECIPHer), School of Social Sciences, CardiffUniversity, Cardiff, UKFull list of author information is available at the end of the article© The Author(s). 2016 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, andreproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link tothe Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.Young et al. BMC Medical Research Methodology (2016) 16:169 DOI 10.1186/s12874-016-0270-8BackgroundThe Health Behaviour in School-aged Children (HBSC)study is a cross-national research project carried out incollaboration with the World Health Organization(WHO). Founded in 1982, the study initially aimed tounderstand smoking behaviours in England, Finland andNorway. The HBSC study is now undertaken every fouryears in over 40 countries and regions across Europeand North America and has expanded to cover a widerange of indicators including health behaviours, riskbehaviours and wellbeing [1]. The study also exploresthe social and developmental context in which youngpeople live, along with their socio-economic conditions.The study uses an anonymous, pen-and-paper, self-complete, internationally standardised questionnaire;administered in classrooms to gain the perspectives of arepresentative proportion of 11, 13 and 15 year oldschool-going children. In some countries, the protocolis also extended to other age groups. The study seeks toadvance the scientific field of adolescent health inter-nationally whilst acting as a monitoring tool to informand influence health services and health educationpolicy and practice at local, national and internationallevels [2].The importance of researching adolescent sexual healthSexually active young people aged 15–24 years are at agreater risk of experiencing adverse health and socialoutcomes including unplanned pregnancy, early mater-nity, abortion or sexually transmitted infections (STIs)than older sexually active populations [3–5]. Early sexualinitiation, inconsistent condom use and multiple sexualpartners are recognised risk factors of STI transmissionand unplanned pregnancy [6, 7]. Reproductive and sexualhealth, including the promotion of safe, healthy, positivesexual behaviour and reproductive choice, therefore re-main a salient public health issue which warrants consist-ent monitoring and more detailed exploration [6, 8, 9].Addressing the sexual health of young people by raisingtheir commitment to safer sex has become a major issueamong developed countries [10–12].The inclusion of sexual health questions in the HBSC StudyFew cross-national studies have been conducted to ex-plore the sexual behaviour and contraceptive use ofyoung people [13, 14], and rarely using comparablequestions, methods or representative samples. Althoughthe first questions relating to sexual health were in-cluded in the HBSC survey in 1989/1990, it was notuntil 2001/2002 that four standardised sexual health andbehaviour questions were included. Derived from the USYouth Risk Behavior Surveillance study (YRBS) [15–17]and the 1986 Minnesota Adolescent Health survey [18],the questions measured experience of sexual intercourse,the age of sexual initiation, methods used to preventpregnancy at last intercourse and condom use at lastintercourse (Fig. 1). Two questions measured condomuse, so as to represent two separate dimensions, one be-ing pregnancy prevention and the other STI prevention.The sexual health questions were designed to bemandatory (i.e., included in all countries or regions). How-ever, each country or region was given the opportunity toopt-out of including these questions in extremis (i.e., if thesensitivity or inappropriateness of the items was likely tojeopardise the completion of the whole survey). In aneffort to reduce such circumstances, the sexual healthquestions were recommended for administration only to15-year-old students [19–24]. At this stage, 35 countriesand regions included the sexual health items with onlyfour refraining from using any of the questions; Denmark,Ireland, Norway and the United States. Data from Malta,Russia, Italy and the Czech Republic were excluded fromanalyses at this time due to deviations from the researchprotocol [21].The 2005/2006 HBSC study included the same sexualhealth questions as those used in 2001/2002. Of the 41participating countries, data were collected from 30countries on experience of sexual intercourse, 31 oncontraceptive pill use, and 30 on condom use [23, 25].In the 2009/2010 HBSC survey, the same four questionswere included as mandatory under the same conditions.Of the 41 participating countries, data were collectedfrom 36 countries on experience of ever having sexualintercourse, 34 countries on contraceptive pill use, and32 countries on condom use [24].Previous work using the HBSC sexual health itemsThe inclusion of standardised, mandatory sexual healthquestions in the HBSC surveys resulted in the publica-tion of internationally comparable data on sexual initi-ation, [20, 22, 26], contraceptive use [23, 25, 27, 28],gender differences, sexual risk behaviour (e.g., non-contraceptive use), socio-economic inequalities and timetrends analysis [24, 29]. Data from the HBSC study arenot only used at an international level, but are also usedto explore various aspects of young people’s sexualhealth and behaviour at a national level [30, 31]. Thesefindings are of significant importance to professionalsinvolved in the care of at-risk adolescents.Problems with the sexual health questionsDespite the inclusion of standardised, mandatory sexualhealth questions, considerable challenges remained in theinterpretation of the international sexual health data.Some countries have introduced the use of skip patternsto avoid students completing questions which are notapplicable. Although there are benefits associated, this hasled to inconsistencies in the quality of the data collected.Young et al. BMC Medical Research Methodology (2016) 16:169 Page 2 of 12Missing data currently poses the biggest issue for thecollection of information about young people’s sexualhealth. Potentially sensitive, the questions are conse-quently only asked to participants aged 15 years andolder. At the beginning of the questionnaire, participantsare reminded that their responses are both anonymousand confidential, and they are encouraged to answerquestions individually. Nevertheless, young people mayrefuse to answer the questions for reasons related to cul-tural norms, religious barriers, embarrassment or peerpressure, thus generating missing data. Many studentswho reported having engaged in sexual intercourse didnot respond to subsequent questions relating to contra-ceptive methods also generating missing data. Further,the layout of the questions (Fig. 1) implies that partici-pants who have not engaged in sexual intercourse had toreport this four times, increasing the potential for frus-tration and stigmatisation.In addition to the potential sensitivity of the questions,a substantial number of inconsistencies had been identi-fied whereby respondents’ answers contradicted theirresponses in previous questions (e.g., reporting condomFig. 1 Standardised questions on sexual health in HBSC 2001/2002 surveyYoung et al. BMC Medical Research Methodology (2016) 16:169 Page 3 of 12use, but also previously reporting that they had neverhad sexual intercourse). Similarly, some participants’responses contradicted answers within the contraceptiveuse questions (e.g., reporting ‘no method of contracep-tion was used’ and then reporting ‘yes’ to a subsequentform of contraception). Inconsistencies were also identi-fied on the two separate questions addressing condom use(i.e., as a method to prevent pregnancy or as a method toprevent STI transmission).Inconsistent, inaccurate, and missing data have poseddifficulties for the comparison of data across HBSC surveyrounds and across countries. Efforts to maintain continu-ity across countries and time periods have been made bythe generation of an internationally comparable data fileby the HBSC Data Management Centre. Countries whohave asked questions that significantly deviate from theHBSC protocol are removed from the data set for thoseparticular questions. There remained however some sub-jectivity about the further development and use of data ata country level, even if recoding syntaxes are centrallyprovided as guidance. This variability led to concernsabout the reliability and comparability of these items.In addition to these methodological issues, broaderethical and pedagogical issues have also been raised relat-ing to the methods of contraception stated as pregnancyprevention. Concerns arose relating to the implied state-ment that withdrawal is an adequate method of pregnancyprevention. This issue may not be in isolation; in order toensure the questionnaire is culturally relevant to youngpeople, and to match some national programs, somecountries add ‘national items’. Seven countries added‘national items’ which included contraceptive methodsconsidered inappropriate for this age group (e.g., with-drawal or natural/biological methods).The issues associated with the sexual health questionsacross the past waves of the HBSC study promptedimportant changes to the instruments measuring sexualhealth and behaviour. The changes are presented belowalong with the findings from a cross-national pilot studyof these proposed changes.Proposed changes to the sexual health questionsThree mandatory sexual health items are included in the2013/2014 HBSC survey. To maintain comparability, theseare largely similar to the mandatory questions from the2001/2002, 2005/2006 and 2009/2010 surveys. The ques-tions still measure participants’ experience of sexual inter-course, age at first intercourse, and the use of contraceptionat last sexual intercourse (condom, birth control pills andother methods) and are still only asked of students aged 15years old and older. The main changes for the 2013/2014international mandatory sexual health questions are theaddition of a skip pattern and alterations to the layout of thecontraceptive methods question.A skip pattern has been introduced after the first ques-tion “Have you ever had sexual intercourse (sometimesthis is called “making love,” “having sex,” or “going allthe way”)?” Respondents who have not had sexual inter-course are directed to the next applicable question. Theresponse option “I have never had sexual intercourse”was removed from subsequent questions. This skip pat-tern simplifies coding, reduces inconsistencies acrossitems, increases comparability between digital and offlinesurveys,1 and reduces stigmatisation and frustration ofrespondents who have not engaged in sexual intercourse.The second change was an alteration to the layout of thecontraceptive methods question. To avoid the inconsist-encies observed in earlier surveys, separate questionswere developed to measure each method of contracep-tion and one sole question measuring condom use wasintroduced (Fig. 2). Finally, in order to overcome ethicaland pedagogical limitations, the question wording wasaltered to remove the connotations that national optionssuch as ‘withdrawal’ were a form of pregnancy preven-tion. Instead, the questions read “The last time you hadsexual intercourse, did you or your partner use” followedby the given method.AimThe aim of the pilot was to provide guidance to theinternational network about the following dimensions ofthe proposed items; answerability (including the extentof missing and inconsistent data), understandability,acceptability (including in different cultures and bothgenders) and relevance to potential participants. Thefindings have been used to guide the inclusion of itemsfor the 2013/2014 HBSC study.MethodsPilot study of proposed changesA pilot of the proposed new mandatory items was con-ducted between 2012 and 2013 in five countries; France(n = 76), Hungary (n = 188), Ireland (n = 228), Portugal(n = 30) and Romania (n = 90), representing a wide rangeof cultural backgrounds. Ethical approval was soughtfrom the University Ethics Boards or other authoritiesassociated with the research team in each country.Schools were recruited based on a purposive conveni-ence sampling method to reflect a range of public, pri-vate, rural and urban schools, and those with varyinglevels of family and socio-economic backgrounds. Classselection was left at the discretion of the school, basedon the age range of participants required for the pilotstudy. All young people in the selected classes wereinvited to participate. Table 1 illustrates the characteris-tics and distribution of schools and classes recruitedwithin each country. Finally, 612 students aged between15–17 years (mean age = 15.55 years, median age = 16Young et al. BMC Medical Research Methodology (2016) 16:169 Page 4 of 12years, SD = 0.95) from public, private, rural and urbanschools from five countries were involved in the pilot.Parental consent was obtained based on the discretion ofthe school (active or passive depending on each country’sschool policies). All participants were asked to provide activeinformed consent prior to participation. The mixed methodspilot study involved a questionnaire administered to stu-dents followed by a qualitative classroom exploration. Allquestionnaires contained the proposed items relating to ex-perience of sexual intercourse, contraception use at lastintercourse and age at first intercourse. Additional questionsrelating to sexual behaviour were asked during the pilotingprocess. These included experience of romantic relation-ships, use of contraception at first intercourse, age of partnerat first intercourse, use of alcohol/other substances at firstintercourse and perception of timing of first intercourse (i.e.,Fig. 2 Standardised questions on sexual health in HBSC pilot surveyYoung et al. BMC Medical Research Methodology (2016) 16:169 Page 5 of 12did you want it to happen at that time?). All questionsunderwent a translation and back-translation process, in ac-cordance with the HBSC international protocol, to ensureinternational consistency and comparability. In Ireland andFrance, additional questions on bullying, alcohol use be-haviour and self-harm were also piloted. All participantswere asked socio-demographic questions. At the endof the questionnaire participants were asked “whenyou filled in the questions, did you think…?”, with re-sponse options “none,” “some,” “most,” or “all of thequestions are inappropriate for young people yourage.” An open-text feedback box at the end of thequestionnaire was also provided, reading “In the boxbelow you can say anything you like about these ques-tions. Tell us if you understood the questions or ifyou feel they are okay to ask young people your age.Remember it is totally anonymous and nobody exceptthe researchers will see what you write.” Prior to fill-ing in the questionnaire participants were assured thatthey did not have to answer any questions that theyTable 1 The distribution and characteristics of schools and classes recruited to take part in the pilot study within each countryNumber of respondents in class(Class n)1 2 3 4 5 6FranceMean = 14.93,Median = 15.00,SD = 1.08F = 59.2%,n = 45School 1Urban public junior high schoolMean = 15, Median = 15,SD = .000, F = 66.7%, n = 812School 2Urban private vocational high schoolMean = 16.62, Median = 17SD = .506, F = 38.5, n = 59 4School 3Rural public junior high schoolMean = 14.49, Median = 14SD = .857, F = 62.7, n = 3215 19 17HungaryMean = 15.77,Median = 15.00,SD = .92F = 56.4%,n = 106School 1Suburban public high schoolMean = 15.77, Median = 15SD = .924, F = 56.4, n = 10628 33 33 32 28 34IrelandMean = 15.92,Median = 16.00,SD = .55F = 37.7%,n = 86School 1Urban mixed gender schoolMean = 16.59, Median = 17SD = .568, F = 48.3, n = 1418 11School 2Urban boys schoolMean = 15.85, Median = 16SD = .428, F = 0, n = 014 23 20 19 22 19School 3Urban girls schoolMean = 15.82, Median = 16SD = .459, F = 100, n = 3418 16School 4Urban girls schoolMean = 15.65, Median = 16SD = .485, F = 100, n = 3434School 5Urban mixed vocational schoolMean = 16.07, Median = 16SD = .730, F = 28.6, n = 414PortugalData not availableSchool 1Urban public high school30RomaniaMean = 14.69,Mean = 15.00,SD = .53F = 58.9%,n = 53School 1Urban public high schoolMean = 14.69, Median = 15SD = .533, F = 58.9, n = 5311 26 29 24Young et al. BMC Medical Research Methodology (2016) 16:169 Page 6 of 12were uncomfortable with and were encouraged tohighlight this issue in their feedback if it was the case.Based on shared guidelines, qualitative data collectionfollowed the individual completion of the questionnaire. Par-ticipants were first asked to underline any words/phraseswhich were difficult to understand and to provide writtenfeedback about each item and/or items they found problem-atic or inappropriate. Subsequently participants were invitedto participate in a collective class discussion. Participantswere first given a chance to have an open discussion aboutthe questionnaire. They were then asked to discuss whetherthey felt that the overall topic was acceptable, relevant andappropriate to young people their age. Each item was thenconsidered individually, and young people were asked openresponse questions as to whether they felt the questions wereappropriate, relevant and what was able to be understood byyoung people their age. Quantitative data were collectedfrom France, Hungary, Ireland and Romania (not Portugal)with a total of 582 participants. Qualitative data were col-lected from all countries with a total of 612 participants.ResultsAnswerabilityWhen asked if they had ever engaged in sexual inter-course, 19.2% of the participants (n = 112) indicated“yes”. A small proportion (n = 19; 3.3%) did not providean answer to this question.Table 2 illustrates the response rates and missing valuesfor reported contraception use at last intercourse amongyoung people who reported having had sexual intercourse.Response rates were very high; prevalence of missing valuesranged from 2.7 to 6.3%. No significant gender or countrylevel differences were identified in the proportion of respon-dents or missing data for any of the contraceptive questions.Table 3 illustrates the reported contraceptive use at lastintercourse among the young people who reported havinghad sexual intercourse. Of those participants who reportedhaving sexual intercourse and answered the question oncondom use (n = 109), 66.1% reported condom use, 28.6%reported that a condom was not used and 2.7% did notknow whether they or their partner used a condom at lastintercourse. Three participants who reported sexual initi-ation (2.7%) did not answer this question.Of the participants who reported experience of sexualintercourse and answered the question on birth control pilluse (n = 107), 17% reported contraceptive pill use at lastintercourse, 62.5% reported no use of birth control pillsand 16.1% of respondents did not know if birth control pillswere used at last intercourse. Five participants who re-ported sexual initiation (4.5%) did not answer this question.Of those participants who reported experience of sexualintercourse and answered the question on other contracep-tive methods use (n = 105), 9.8% reported some othermethod of contraception at last intercourse, 67.0% reportedno other method of contraception was used, and 17.0% didnot know if another method of contraception was used atlast intercourse. Seven participants who reported sexual ini-tiation (6.3%) did not respond to this question.Of those respondents who reported sexual initiation,96.4% (n = 108) reported the age at which they first hadintercourse. Four participants who reported sexual initi-ation (3.6%) did not respond to this question.Despite the skip pattern, some participants gave in-consistent responses; of those participants who re-ported never having engaged in sexual intercourse,two (0.4%) reported “using a condom at last inter-course,” three (0.7%) reported “don’t know” in relationto condom use and three (0.7%) reported “don’t know”in relation to the use of other contraceptive methodsat last intercourse. One participant (0.2%) reportedusing a condom at last intercourse, despite leaving thequestion on experience of sexual intercourse blank.One participant (0.2%) reported using the contracep-tive pill while reporting never having engaged in inter-course. According to the skip logic, this may appearinconsistent; however it is possible that this partici-pant was indeed on the contraceptive pill without hav-ing engaged in sexual intercourse. Finally, threeparticipants (0.7%) reported that they did not know ifbirth control pills were used at last intercourse, whilesaying that they did not have sexual intercourse. Noparticipants who reported never having engaged insexual intercourse reported an age of first intercourse(i.e., inconsistent data).UnderstandabilityDuring the classroom discussion the overwhelming ma-jority of students understood the terms ‘sexual inter-course’ and the colloquial terms used. Students generallyreported no problems understanding questions relatingto contraceptive methods or the age of first intercourse.When asked to provide “general comments” in the opentext box at the end of the questionnaire, 219 respon-dents (37.6%) provided responses. A total of 174 respon-dents (79.4%) recorded that the questions were clearand/or easy to understand.“All the questions are clear, do not change a thing”(Participant, Hungary)Table 2 Response rate and missing values for contraception atlast intercourse among young people who reported having hadsexual intercourse (n = 112)Respondents % (n) Missing % (n)Condom 97.3 (109) 2.7 (3)Pill 95.5 (107) 4.5 (5)Other method 93.8 (105) 6.3 (7)Young et al. BMC Medical Research Methodology (2016) 16:169 Page 7 of 12“This questionnaire is very easy to get”(Participant, France)“Easily understood and straightforward”(Participant, Ireland)Only five respondents (2.3%) reported the questions asunclear or confusing.“I found the ‘now skip to question…’ a bit confusing.The questions are okay” (Participant, Ireland)“Sometimes a bit confusing to follow.”(Participant, Ireland).Although the skip logic reduced the proportion ofmissing and inconsistent data, skip patterns in generalcan create some challenges in surveys of adolescents. Inthis pilot, the small number of participants whoexpressed confusion about the items indicated that theskip pattern instructions were the unclear aspect.Acceptability of sexual behaviour questions andrelevance to participantsOf the 559 respondents to the question relating to the in-appropriateness of the questions, 64.3% reported that noneof the questions were inappropriate while 19.6% reportedthat some of the questions were inappropriate. A total of7.4% reported that most of the questions wereinappropriate, whereas 4.5% reported that all of the ques-tions were inappropriate. Chi-square tests identified sig-nificant cross-national differences in the responses (χ2 =22.831, df = 9, p = .007). Young people across all four re-gions were similar in the proportion of those reportingthat “some” or “all” of the questions are age-inappropriate.Fewer Romanian participants indicated that they found“none” of the questions inappropriate, whereas fewerHungarian respondents reported that “most” of the itemswere inappropriate. Cramér’s V was used to provide anestimate of the effect size of the association, V = .117,p = .007. This indicates that despite the significant associ-ation between region and ratings of age-inappropriateness,it is a very low effect. No significant association wasfound between the acceptability of the questions andage (χ2 = 5.967, df = 9, p = .743) or gender of partici-pants (χ2 = 2.132, df = 3, p = .545). Cross-cultural oreven school-level differences may confound the asso-ciation, and the results should be treated with cautiondue to the low sample sizes. Given that additionalquestions relating to sexual behaviour were askedduring the piloting process, and that in Ireland andFrance, additional questions on other health behav-iours were also included, it is important to acknow-ledge that comments relating to the appropriatenessof questions may relate to these supplementary ques-tions on the questionnaire, even when the researchersfacilitating the focus group discussion were asked toclearly separate the topics.When reviewing the data from the written commentson the questionnaires, the main concern expressedabout the question asking whether participants had everengaged in sexual intercourse (n = 3) was that the ques-tion was limited to penetrative sexual intercourse.“Most people our age have done everything but sex,just saying” (Participant, Ireland)One participant questioned the usefulness of the explan-ation between the brackets. This was also raised by twostudents in the class discussion in Portugal. Aside fromthese comments, students did not raise written concernsabout the acceptability and relevance of this question.Similarly, no specific comments relating to acceptabilityor relevance were volunteered about the contraceptive useor the age of first intercourse questions. During the classdiscussion some students expressed embarrassment, and asmall minority considered the questions private, howeverthere were no significant objections to the questions, orreports that they were unacceptable or irrelevant to youngpeople this age.Of the 219 participants who provided responses to the“general comments” section at the end of the question-naire, seven said that the questions were private or per-sonal, but did not specify whether this was a problem. Atotal of 25 students stated that the questions were in-appropriate, too personal or private or felt that theywere not adapted to people of their age group (France,n = 1; Hungary, n = 7; Ireland, n = 14; Romania, n = 3).“I feel that some of the questions were quite invasiveand too personal.” (Participant, Ireland)“I want to say that some items are too private andthey should not be used because they refer to yourprivate life. Some students are shyer and they don’ttalk about their life.” (Participant, Romania)“The questions are clear but sometimes childish;teenagers in my age can talk about sex seriously.”(Participant, Hungary)Table 3 Contraception use at last intercourse among theyoung people who reported having had sexual intercourseUse % (n) No use % (n) Do not know % (n)Condom (n = 109) 66.1 (74) 28.6 (32) 2.7 (3)Pill (n = 107) 17.0 (19) 62.5 (70) 16.1 (18)Other method (n = 105) 9.8 (11) 67.0 (75) 17.0 (19)Young et al. BMC Medical Research Methodology (2016) 16:169 Page 8 of 12A total of 178 students reported that they felt thequestions were appropriate, suitable for their age orrelevant to young people.“All the questions are clear; you can ask my peersabout the topic” (Participant, Hungary)“These questions were perfect to present to teenagers.They tackle all important issues in a teenagers life,and they were really well laid out and clear”(Participant, Ireland)“Your questionnaire is well done, adapted to theteenagers” (Participant, France)A small number of students (n = 13) raised concernsabout whether students would answer honestly.“Some people are not going to say if they did some ofthe things mentioned” (Participant, Ireland)“The questions are clear, but I don’t think everyoneanswered honestly” (Participant, Hungary)A total of 28 participants provided suggestions for add-itional, more detailed sexual health/behaviour questionsthat they felt would also be important to young people(e.g., relating to virginity, wanting to have sex, other sexualbehaviours and different types of relationships).“It could be more profound, and should refer to topicssuch as sex education or petting” (Participant, Hungary).“Should have more questions for virgins”(Participant, Hungary)“Should include more about boy-girl friendships”(Participant, Ireland)ReliabilityA test-retest study was carried out in three schools inIreland consisting of a total of 63 participants. Kappastatistics were calculated for items measuring engagementin sexual intercourse (κ = 0.832), age of sexual initiation(κ = 0.788) and condom (κ = 0.730), birth control pill (κ =0.781) and other contraceptive (κ = 0.451) use at last inter-course. Apart from the question on other contraceptiveused at last intercourse, all contained a sufficient level ofinter-rater reliability (i.e., kappa above 0.6) indicating theirsuitability for use in the HBSC survey.DiscussionAn international pilot study conducted in five countriessuggests that despite some small cross-national differencesin the self-reported quantitative data relating to the appro-priateness of the questions, the majority of participants ineach country reported positive feedback about the appro-priateness of the questions. Qualitative feedback suggeststhat, overall, the phrasing and age-targeting of the ques-tions were considered suitable by the vast majority ofstudents. With the exception of a small number of respon-dents who commented on the clarity and/or personalnature of the content, no specific issues with the questionswere identified.Missing and inconsistent data posed two of the mostsignificant issues with previous HBSC survey rounds.The newly introduced skip pattern and alterations to thelayout of the contraceptive methods question have re-duced the potential for participants to provide missingand inconsistent data. When comparing the consistent,inconsistent and missing responses from the 2009/10HBSC study data (limited to the four target countries)and the 2013-pilot data, the proportion of the studentswho responded in an inconsistent way has reduced, al-though not significantly across the study sweeps (1.4% in2010 and 1.0% in the 2013 pilot). However, simplifica-tion of the item structure resulted in a significantdecrease in the number of missing responses, from 6.5to 3.1% (χ2 = 29.651, df = 3, p < .001). This positively im-pacts on data quality for future HBSC survey rounds ofdata collection. Some countries have also introduceddigital questionnaire completion. This would furtherreduce the problem of inconsistent responses, prevent-ing those who do not report having had sexual inter-course from spending time on any related questions.Online testing would also simplify the skip questioninstructions mentioned by some as a problem. Thisbenefit would go beyond the sexual health questions ifintroduced as a future HBSC standard.For practical, political and ethical reasons, there hasbeen difficulty including the sexual health questions inthe HBSC study in some countries. The current findingsindicate that young people from five culturally differentcountries, on the whole, are open to answering anonym-ous questions relating to their sexual behaviour. Thevast majority of feedback, in relation to acceptability andrelevance of these questions, was positive. Even in thosecountries where the sexual health questions are includedin the HBSC survey, asking young people questionsabout sexual health and behaviour may be potentiallyembarrassing or even annoying. For this reason, it iscrucial to strive for the most simple and neutral questionformat. As a result of the pilot study feedback, changesto the pilot have ensured that participants are remindedthat they can choose not to answer any questions thatthey do not want to answer.Safe and healthy sexual behaviour, including reproduct-ive choice, not only play a key role in young people’sYoung et al. BMC Medical Research Methodology (2016) 16:169 Page 9 of 12health and wellbeing, but are also salient public healthissues. With many young people initiating sexual inter-course and engaged in risk behaviours during adolescence,it is a crucial time to measure young people’s sexual be-haviours as a way to inform and influence health services,and health education policy and practice at local, nationaland international levels. There is currently very limitedhigh-quality, cross-national data available which exploresthe sexual behaviour and contraceptive use of youngpeople. The HBSC study therefore provides a uniqueopportunity to collect high quality, internationally com-parable data not only about the sexual health behavioursof young people, but also about the context in whichyoung people live and develop.Methodological, ethical and pedagogical issues hadhowever been raised relating to the sexual health itemsintroduction on a mandatory basis to the HBSC study in2002. These issues have prompted important changes tothe questions measuring sexual behaviour; the additionof a skip pattern and alterations to the layout of thecontraceptive methods question. The results have pro-vided guidance on the answerability (including theextent of missing and inconsistent data), understandabil-ity, acceptability (including in different cultures) andrelevance to potential participants.The pilot study is not however without limitations.First it should be acknowledged that the sample size isquite small and not representative at country level, des-pite an effort to collect data in contrasting schools.Completion of the questionnaire required self-reportdata from young people about the potentially sensitivetopic of sexual behaviour. Although participants com-pleted questionnaires both anonymously and confiden-tially, and every effort was made to ensure that youngpeople completed the questionnaire individually (i.e.,without any influence from their peers), the limitationsof collecting self-report data about sensitive topics hasbeen well documented [32]. In addition, whilst partici-pants were given the opportunity to provide writtenfeedback about their acceptability and understanding ofthe questions, they may have been unwilling to disclosetheir lack of understanding or embarrassment of thequestions verbally in the non-anonymous peer-groupgroup discussion.Another limitation is the current wording of the ques-tion relating to sexual intercourse. It has been shown thatwith the help of the information given in parenthesis,young people interpret this as penetrative vaginal sex only[33, 34], however no anatomical definition of sexual inter-course is provided. This also leaves aside other forms ofsexual behaviour that could be an STI risk. Additionalquestions were included in the pilot study that looked atexperience of romantic relationships, including a questionthat explored same-sex relationships. It is hoped that theinclusion of these questions would prevent young peoplefrom feeling marginalised by any heteronormative ques-tions, and although they may not report feelings of mar-ginalisation in the class discussion, any comments couldbe made when instructed to write on the questionnaire.Finally, it must also be acknowledged that the pilotquestionnaire contained additional questions relating toromantic relationships and the circumstances surround-ing first sexual intercourse (contraception, age of part-ner, use of substances), and in Ireland and France,further additional questions on bullying, alcohol use andself-harm. The evaluation of the sexual health items mayalso partially reflect an evaluation of these questions orbe altered by the evaluation of these questions.ConclusionTo summarise, following the issues highlighted in rela-tion to the mandatory sexual health questions includedin the HBSC survey since 2002, changes have been madeto the HBSC survey questions. The core meaning of thequestions has not changed, but findings from the pilotstudy suggest that the simplification has significantlyreduced the proportion of missing data, thus enhancingthe capacity of the questions to measure sensitiveaspects of young people’s sexual behaviour cross-nationally. As such the adapted questions have beenincluded in the 2013/2014 round of the HBSC survey.The Sexual Health Focus Group within the HBSCResearch Network will continue to monitor trends inadolescent sexual health and behaviours and improvetheir measures, with the goal to inform and influencehealth services, decision makers in health educationpolicy and all professionals caring for adolescents atlocal, national and international levels.Endnote1A number of countries had already introduced skippatterns into their questions, but this has not beenapplied consistently across the international methodo-logical design.AbbreviationsHBSC: Health behaviour in school-aged children; HIV: Human immunodeficiencyvirus; STI: Sexually transmitted infection; STIs: Sexually transmitted infectionsAcknowledgementsThe authors thank all the schools and young people who participated in thepilot study and who have participated in past HBSC studies. They would alsolike to thank the following colleagues for their help with the pilot study:Ágnes Németh (Principal Investigator) and Emese Zsiros from Hungary, TheHBSC Ireland Team, the Addiction Switzerland Team, Adriana Baban(Principal Investigator, Romania), Margarida Gaspar de Matos (PrincipalInvestigator) and Lúcia Ramiro from Portugal. HBSC is an international studycarried out in collaboration with WHO/EURO. The International Coordinatorof the 2013/2014 survey was Professor Candace Currie, PhD (University of St.Andrews, School of Medicine, Child and Adolescent Health Research Unit,Scotland) and the Data Bank Manager was Professor Oddrun Samdal, PhD(University of Bergen, HBSC Data Management Centre, Norway). For detailsYoung et al. BMC Medical Research Methodology (2016) 16:169 Page 10 of 12of the HBSC study, see http://www.hbsc.org. The authors would also like tothank all past and current members of the HBSC Sexual Health Focus Group.Finally, they thank the valuable comments of the reviewers on the firstversion of the manuscript.FundingHBSC France is funded by INPES (National institute for Health Promotion andEducation) and OFDT (French Monitoring Centre for Drug Use andAddiction).HBSC Hungary is funded by former OGYEI (National Institute of Child Health)(currently NEFI, National Institute of Health Promotion).HBSC Ireland is funded by the Department of Health.HBSC Portugal is funded by DGS (Director General of Health, Ministry of Health).HBSC Romania is funded by the Norway Innovation Grants through theRomanian Ministry of Health.Availability of data and materialsThe datasets supporting the conclusions of this article are available viathe Health Behaviour in School-aged Children repository, contactable athttp://www.hbsc.org/.Authors’ contributionsThe original idea for the paper was generated at the Sexual Health FocusGroup meeting in Toulouse 2013. EG, SNG, ES, NM, BW, AK and HYconceptualised the paper. The pilot study to test 2013/2014 HBSC questionsof sexual health items was conducted by EG (France), AK (Hungary), SNG, HYand LB (Ireland), MR (Portugal), APC (Romania) and their colleagues. BWconstructed the database for the pilot study. HY led on the analysis andinterpretation of the data with significant contribution from AK, MR and ES.HY wrote the first draft of the manuscript. All authors contributed tosubsequent drafts and approved the final manuscript.Competing interestsThe authors declare that they have no competing interests.Consent for publicationAll identifiable information has been removed from the publication.Ethics approval and consent to participateEthical approval was sought from the University Ethics Boards or otherauthorities associated with the research team in each country; in France, theFrench National Commission on Informatics and Liberty (CNIL); in Hungary,The Hungarian Medical Scientific Council; in Ireland, The Human ResearchEthics committee of the National University of Ireland Galway; in Portugal,The Hospital São João do Porto; in Romania, the Ethical Committee from theBabes Bolyai University, Cluj Napoca. Parental consent was sought at thediscretion of the school. Informed consent was sought from participantsprior to participation.Author details1Centre for the Development and Evaluation of Complex Interventions forPublic Health Improvement (DECIPHer), School of Social Sciences, CardiffUniversity, Cardiff, UK. 2National Institute of Health Promotion, Budapest,Hungary. 3Eötvös Loránd University, Institute of Psychology, Budapest,Hungary. 4Aventura Social - Faculdade de Motricidade Humana, Universidadede Lisboa, [University of Lisbon], Lisbon, Portugal. 5ISAMB/Faculdade deMedicina da Universidade de Lisboa [Faculty of Medicine, University ofLisbon], Lisbon, Portugal. 6FCT - Fundação para a Tecnologia e Ciência[Foundation for Science and Technology] (SFRH/BPD/110905/2015), Lisbon,Portugal. 7University of British Columbia, School of Nursing, Vancouver,Canada. 8Université Libre de Bruxelles (ULB), Service d’Information PromotionEducation Santé (SIPES), School of Public Health, Brussels, Belgium. 9HealthPromotion Research Centre, National University of Ireland, Galway, Ireland.10Child and Adolescent Health Research Unit, School of Medicine, Universityof St Andrews, St Andrews, Scotland, UK. 11Addiction Switzerland, ResearchDepartment, Lausanne, Switzerland. 12Service Médical du Rectorat del’académie de Toulouse, UMR 1027 Inserm, Université Paul Sabatier, Toulouse,France.Received: 11 February 2016 Accepted: 23 November 2016References1. Currie C, Inchley J, Molcho M, Lenzi M, Veselska Z, Wild F, editors. HealthBehaviour in School-aged Children (HBSC), Study Protocol: Background,Methodology and Mandatory Items for the 2013/14 Survey. St. Andrews: Childand Adolescent Health Research Unit (CAHRU), University of St. Andrews; 2014.2. Currie C, Nic Gabhainn S, Godeau E. The Health Behaviour in School-agedChildren: WHO Collaborative Cross-National (HBSC) Study: origins, concept,history and development 1982–2008. Int J Public Health. 2009;54:131–9.3. Centers for Disease Control and Prevention (CDCP). Sexually transmitteddisease surveillance 2010. 2010. http://www.cdc.gov/std/stats10/surv2010.pdf. Accessed 25 July 2013.4. Wellings K, Jones KG, Mercer CH, Tanton C, Clifton S, Datta J, Copas AJ, Erens B,Gibson LJ, Macdowall W, et al. The prevalence of unplanned pregnancy andassociated factors in Britain: findings from the third National Survey of SexualAttitudes and Lifestyles (Natsal-3). Lancet. 2013;382:1807–16.5. Department of Health (England). Abortion statistics for England and Wales,2013. 2013.6. Bearinger LH, Sieving RE, Ferguson J, Sharma V. Global perspectives on thesexual and reproductive health of adolescents: patterns, prevention, andpotential. Lancet. 2007;369:1220–31.7. Gravningen K, Furberg A-S, Simonsen GS, Wilsgaard T. Early sexualbehaviour and Chlamydia trachomatis infection – a population based cross-sectional study on gender differences among adolescents in Norway. BMCInfect Dis. 2012;12:1–11.8. Santelli JS, Lindberg LD, Finer LB, Singh S. Explaining recent declines inadolescent pregnancy in the United States: the contribution of abstinenceand improved contraceptive use. Am J Public Health. 2007;97:150–6.9. Anderson JE, Santelli JS, Morrow B. Trends in adolescent contraceptive use,unprotected and poorly protected sex, 1991–2003. J Adolesc Health. 2006;38:734–39.10. Avery L, Lazdane G. What do we know about sexual and reproductivehealth of adolescents in Europe? Eur J Contracept Reprod Health Care.2008;13:58–70.11. UN. The Millenium Development Goals Report 2010. New York: UnitedNations Development Project; 2006. http://www.un.org/millenniumgoals/pdf/MDG%20Report%202010%20En%20r15%20-low%20res%2020100615%20-.pdf. Accessed 4 Dec 2015.12. WHO. Global strategy for the prevention and control of sexually transmittedinfections: 2006–2015: Breaking the chain of transmission. Geneva: WorldHealth Organisation; 2007. http://apps.who.int/iris/bitstream/10665/43853/1/9789241563475_eng.pdf. Accessed 2 Nov 2015.13. Darroch JE, Frost JJ, Singh S, The Study Team. Teenage sexual andreproductive behavior in developed countries: can more progress be made?Occasional Report No 3. New York: Alan Guttmacher Institute; 2001. https://www.guttmacher.org/sites/default/files/pdfs/pubs/euroteens_summ.pdf.Accessed 2 Jan 2016.14. Berne L, Huberman B. European approaches to adolescent sexual behaviorand responsibility. Washington, DC: Advocate for Youth; 2001.15. Kolbe LJ, Kann L, Collins JL. Overview of the youth risk behavior surveillancesystem. Public Health Rep. 1993;108 Suppl 1:1–10.16. Brener ND, Grunbaum JA, Kann L, McManus T, Ross J. Assessing health riskbehaviors among adolescents: the effect of question wording and appealsfor honesty. J Adolesc Health. 2004;35:91–100.17. Grunbaum JA, Kann L, Kinchen SA, Williams B, Ross JG, Lowry R, Kolbe L.Youth Risk Behavior Surveillance — United States, 2001. J School Health.2002;72:313–28.18. Blum R, Harris LJ, Resnick MD, Rosenwinkel K. Technical report on theadolescent health survey, University of Minnesota Adolescent HealthProgram. Minneapolis: University of Minnesota; 1989.19. Ross J, Wyatt W. Sexual behaviour. In: Currie C, Hurrelman W, SettertobulteW, Smith R, Todd J, editors. Health and health behaviour among youngpeople: Health Behaviour in School-aged Children (HBSC): A WHO cross-national study: International Report from the 1997/1998 survey.Copenhagen: WHO Regional Office for Europe; 2000.20. Ross J, Godeau E, Dias S. Sexual health in young people—Findings from theHBSC study. Entre Nous. 2004;58:20–3.21. Ross J, Godeau E, Dias S. Sexual health. In: Currie C, Roberts C, Morgan A,Smith R, Settertobulte W, Samdal O, Barnekow Rasmussen V, editors. YoungPeople’s Health in Context, Health Behaviour in School-aged Children study:International Report from the 2001/2002 Survey Health Policy for Childrenand Adolescents Volume No. 4. Copenhagen: WHO Regional Office forEurope; 2004. p. 153–60.Young et al. BMC Medical Research Methodology (2016) 16:169 Page 11 of 1222. Ross J, Godeau E, Dias S, Vignes C, Gross L. Setting politics aside: sexual healthin young people: findings from the HBSC study. SIECUS Rep. 2004;32:28–34.23. Currie C, Nic Gabhainn S, Godeau E, Roberts C, Smith R, Currie D, Pickett W,Richter M, Morgan A, Barnekow V. Inequalities in young people’s health:international report from the HBSC 2005/06 survey. Health Policy for Childrenand Adolescents, vol. 5. Copenhagen: WHO Regional Office for Europe; 2008.24. Currie C, Zanotti C, Morgan A, Currie D, de Looze M, Roberts C, Samdal O,Smith ORF, Barnekow V. Social determinants of health and well-being amongyoung people. Health Behaviour in School-aged Children (HBSC) study:international report from the 2009/2010 survey. Health Policy for Children andAdolescents, vol. 6. Copenhagen: WHO Regional Office for Europe; 2012.25. Nic Gabhainn S, Baban A, Boyce W, Godeau E, the HBSC Sexual Health FocusGroup. How well protected are sexually active 15-year olds? Cross-nationalpatterns in condom and contraceptive pill use 2002–2006. Int J Public Health.2009;54:209–15.26. Woynarowska B, Izdebski Z, Kołoło H, Mazur J. Sexual initiation and use ofcondoms and other methods of contraception among 15-year-oldadolescents in Poland and other countries. Ginekol Pol. 2004;75:621–32.27. Godeau E, Nic Gabhainn S, Vignes C, Ross J, Boyce W, Todd J. Contraceptiveuse by 15-year-old students at their last sexual intercourse: Results from 24countries. Arch Pediatr Adolesc Med. 2008;162:66–73.28. Lazarus JV, Moghaddassi M, Godeau E, Ross J, Vignes C, Östergren PO,Liljestrand J. A multilevel analysis of condom use among adolescents in theEuropean Union. Pub Health. 2009;123:138–44.29. Ramiro L, Windlin B, Reis M, Gabhainn SN, Jovic S, Matos MG, Magnusson J,Godeau E. Gendered trends in early and very early sex and condom use in20 European countries from 2002 to 2010. Eur J Pub Health. 2015;25:65–8.30. Godeau E, Vignes C, Duclos M, Navarro F, Cayla F, Grandjean H. Facteursassociés à une initiation sexuelle précoce chez les filles : données françaisesde l’enquête internationale Health Behaviour in School-aged Children(HBSC)/OMS [Associated factors to early sexual initiation among girls:French data from the international Health Behaviour in School-agedChildren (HBSC)/OMS]. Gynecol Obstet Fertil. 2008;36:176–82.31. Jovic S, Delpierre C, Ehlinger V, Sentenac M, Young H, Arnaud C, Godeau E.Associations between life contexts and early sexual initiation among youngwomen in france. Perspect Sex Reprod Health. 2014;46:31–9.32. Fenton KA, Johnson AM, McManus S, Erens B. Measuring sexual behaviour:methodological challenges in survey research. Sex Transm Infect. 2001;77:84–92.33. Brener ND, Kann L, McManus T, Kinchen SA, Sundberg EC, Ross JG.Reliability of the 1999 Youth Risk Behavior Survey questionnaire. J AdolescHealth. 2002;31:336–42.34. Orr DP, Fortenberry DJ, Blythe MJ. Validity of self‐reported sexualbehaviors in adolescent women using biomarker outcomes. Sex TransmDis. 1997;24:261–6.• We accept pre-submission inquiries • Our selector tool helps you to find the most relevant journal• We provide round the clock customer support • Convenient online submission• Thorough peer review• Inclusion in PubMed and all major indexing services • Maximum visibility for your researchSubmit your manuscript atwww.biomedcentral.com/submitSubmit your next manuscript to BioMed Central and we will help you at every step:Young et al. BMC Medical Research Methodology (2016) 16:169 Page 12 of 12"@en ; edm:hasType "Article"@en ; edm:isShownAt "10.14288/1.0340075"@en ; dcterms:language "eng"@en ; ns0:peerReviewStatus "Reviewed"@en ; edm:provider "Vancouver : University of British Columbia Library"@en ; dcterms:publisher "BioMed Central"@en ; ns0:publisherDOI "10.1186/s12874-016-0270-8"@en ; dcterms:rights "Attribution 4.0 International (CC BY 4.0)"@en ; ns0:rightsURI "http://creativecommons.org/licenses/by/4.0/"@en ; ns0:scholarLevel "Faculty"@en ; dcterms:subject "Adolescent sexual health"@en, "Adolescent sexual behaviour"@en, "Self-completion"@en, "Questionnaire design inconsistencies"@en, "Missing data"@en ; dcterms:title "Sexual Health questions included in the Health Behaviour in School-aged Children (HBSC) Study: an international methodological pilot investigation"@en ; dcterms:type "Text"@en ; ns0:identifierURI "http://hdl.handle.net/2429/59885"@en .