UBC Faculty Research and Publications

Deriving and validating a risk estimation tool for screening asymptomatic chlamydia and gonorrhea Falasinnu, Titilola; Gilbert, Mark; Gustafson, Paul; Shoveller, Jean 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-Derivation and Temp validity.pdf [ 604.45kB ]
JSON: 52383-1.0228252.json
JSON-LD: 52383-1.0228252-ld.json
RDF/XML (Pretty): 52383-1.0228252-rdf.xml
RDF/JSON: 52383-1.0228252-rdf.json
Turtle: 52383-1.0228252-turtle.txt
N-Triples: 52383-1.0228252-rdf-ntriples.txt
Original Record: 52383-1.0228252-source.json
Full Text

Full Text

This version of the article “Falasinnu T, Gilbert M, Gustafson P, Shoveller, J. Deriving and validating a risk estimation tool for screening asymptomatic chlamydia and gonorrhea. Sex Transmitted Diseases. 2014;41(12):706. doi: 10.1097/OLQ.0000000000000205.” is not the final version.   We acknowledge the publisher’s (Wolters and Kluwer) copyright.  Here is a link to the final published version of the article: http://www.ncbi.nlm.nih.gov/pubmed/25581805 ARTICLE COVERSHEETLWW_CONDENSED(7.75X10.75)SERVER-BASEDArticle : OLQ14083Creator : bbatigulaoDate : Monday October 6th 2014Time : 10:15:16Number of Pages (including this page) : 9Deriving and Validating A Risk Estimation Tool forScreening Asymptomatic Chlamydia and GonorrheaTitilola Falasinnu, MHS,* Mark Gilbert, MD, MSc,† Paul Gustafson, PhD,‡ and Jean Shoveller, PhD§Background: There has been considerable interest in the developmentof innovative service delivery modules for prioritizing resources in sexualhealth delivery in response to dwindling fiscal resources and rising infec-tion rates.Methods: This study aims to derive and validate a risk scoring algorithmto accurately identify asymptomatic patients at increased risk for chlamydiaand/or gonorrhea infection. We examined the electronic records of patientvisits at sexual health clinics in Vancouver, Canada. We derived risk scoresfrom regression coefficients of multivariable logistic regression modelusing visits between 2000 and 2006. We evaluated the model's discrim-ination, calibration, and screening performance. Temporal validation wasassessed in visits from 2007 to 2012.Results: The prevalence of infection was 1.8% (n = 10,437) and 2.1%(n = 14,956) in the derivation and validation data sets, respectively. Thefinal model included younger age, nonwhite ethnicity, multiple sexual part-ners, and previous infection and showed reasonable performance in thederivation (area under the receiver operating characteristic curve = 0.74;Hosmer-Lemeshow P = 0.91) and validation (area under the receiver oper-ating characteristic curve = 0.64; Hosmer-Lemeshow P = 0.36) data sets.A risk score cutoff point of at least 6 detected 91% and 83% of cases byscreening 68% and 68% of the derivation and validation populations,respectively.Conclusions: These findings support the use of the algorithm for individ-ualized risk assessment and have important implications for reducing un-necessary screening and saving costs. Specifically, we anticipate that thealgorithm has potential uses in alternative settings such as Internet-basedtesting contexts by facilitating personalized test recommendations, stimu-lating health care–seeking behavior, and aiding risk communication by in-creasing sexually transmitted infection risk perception through the creationof tailored risk messages to different groups.BACKGROUNDSexual health care is suffering from a budgetary crisis ashealth care costs escalate in almost every high-income country,and the recent economic crisis exacerbates the financial challengesalready facing publicly funded health care programs.1 Sexualhealth clinics in several jurisdictions are experiencing budget cuts,hiring freezes, and, in some cases, closures; surviving clinics havehad to limit operating hours, cut some services, and streamline op-erations.1 In this context, improving sexual health care delivery isarguably of paramount interest. Moreover, inadequate access tosexual health care has potentially detrimental effects on individ-uals and also the community.2 Thus, it is imperative to adopt sys-tems that optimize the delivery of comprehensive and high qualitysexually transmitted infections (STI) testing services while mini-mizing public health budget demands.3 For example, several juris-dictions have adopted alternative service delivery modules, suchas Internet-based testing and triaging services that involve sex-ual health clients having limited interaction with clinicians.4,5 InBritish Columbia (BC), we are developing Get Checked Online(GCO), an Internet-based STI testing service intended to supple-ment existing face-to-face clinic-based sexual health services withthe goal of increasing testing uptake and frequency and to easedemand on clinic-based testing.4,6One important feature of efficient STI control programs,especially in novel health service delivery models such as GCO,is making certain that those at increased risk for STIs have accessto screening services because identifying and treating infections inthis group can effectively terminate onward transmission andtherefore prevent new cases of disease. However, much is still un-known about how best to maximize access for those at highestrisk, particularly in contexts where STI clinics are overburdenedwith symptomatic clients.2 Also, identifying high-risk asymptom-atic clients may also confer public health benefits as undiagnosedand untreated infections can frequently progress to complicationssuch as pelvic inflammatory disease and infertility (in women)and epididymitis (in men).7 To maximize case finding in asymp-tomatic visits, selective screening (based on risk assessment andentails the screening of individuals who meet prespecified criteria)may be a prudent approach because it minimizes the costs associ-ated with testing low-risk individuals.8The current body of knowledge indicates that risk predic-tion rules that capture a continuous risk spectrum are excellenttools for targeted screening.9 A recent review critically appraisingprediction rules used for sexual health service provision found thatthese have been explored in a variety of contexts including emer-gency departments, population-based settings and STI clinics.10However, there are currently no published prediction rules forscreening asymptomatic chlamydia and/or gonorrhea. The currentunderstanding indicates that increasing testing access for high-riskasymptomatic individuals may result in long-term health andhealth economic impacts that surpass those associated with clini-cally observable infections.7Here, we examine the performance of a selective screeningstrategy (derived from a clinical prediction rule) in identifyingFrom the *The School of Population and Public Health, University ofBritish Columbia, Vancouver, Canada; † British Columbia Center forDisease Control, Vancouver, Canada; ‡The Department of Statistics,University of British Columbia, Vancouver, Canada; and §The Schoolof Population and Public Health, University of British Columbia,Vancouver, CanadaConflict of interest disclosure and sources of funding: Titilola Falasinnu issupported by the Canadian Institutes of Health Research Doctoral Re-search Award. Paul Gustafson is supported by a grant from the NaturalSciences and Engineering Research Council of Canada. All of the re-maining authors have disclosed that they have no financial relation-ships with or interests in any commercial companies pertaining tothis article.CorrespondenceAQ1 : Titilola Falasinnu, MHS, The School of Population andPublic Health, University of British Columbia, 2206 East Mall,Vancouver, BC, Canada V6T 1Z3. E-mail: lola.falasinnu@ubc.ca.Received for publicationMarch 31, 2014, and accepted September 9, 2014.A study of sexual health clinic clients in Vancouver identified a well-performing risk scoring algorithm for screening asymptomatic chlamydiaand gonorrhea infection using demographic and behavioral risk factors.DOI: 10.1097/OLQ.0000000000000205Copyright © 2014 American Sexually Transmitted Diseases AssociationAll rights reserved.ORIGINAL STUDYSexually Transmitted Diseases • Volume 00, Number 00, Month 2014 1Copyedited by: K. Cabrerapersons at increased risk for Chlamydia trachomatis or gonococ-cal infections. Specifically, this study aimed to derive a riskscoring tool for screening for asymptomatic chlamydia and/orgonorrhea infection among patients seen at STI clinics between2000 and 2006 (derivation population) and test the generalizabil-ity of the algorithm in a more recent period among patients seenbetween 2007 and 2012 (temporal validation population).MATERIALS AND METHODSStudy PopulationWe used electronic medical records from asymptomaticpatients tested for chlamydia and/or gonorrhea between 2000and 2012 at 2 STI clinics in Vancouver, BC. All statistical analyseswere performed using SAS version 9.3. We have previously de-scribed the study setting, population, and variables examined.6Briefly, the data were derived from the STI Information System(STI IS), a database that houses risk assessment information andlaboratory results of patient visits at publicly funded STI clinicsin BC. Data from each new client consultation between 2000and 2012 among women and heterosexual men were included.This analysis was limited to asymptomatic clinic visits that arenot sexual contacts of known STI cases as ascertained by the at-tending clinician during risk assessment. Repeat visits within30 days of a previous clinical visit were also excluded to avoidincluding clients receiving confirmatory diagnoses. Chlamydiaand/or gonorrhea infection was measured as a composite outcomebecause most laboratories use multiplex assays that test for bothinfections simultaneously11 and also because the relevant clinicaldecision is whether to offer this test or not. At these clinics, chla-mydia, gonorrhea, syphilis, and HIV tests are generally offered toall sexually active clients at each visit.For this analysis, we extracted a range of demographic andbehavioral information from patient visits such as age, sex, race/ethnicity, number of sexual partners in the previous 6 months, con-dom use, injection drug use (IDU), sex with partners recruited on-line, sex with IDU, sex with commercial sex workers (CSWs),previous diagnosis with chlamydia, and previous diagnosis withgonorrhea. Missing data among predictors were imputed 5 timesusing the Sequential Regression Imputation Method, and thisresulted in estimates that were averaged using Rubin's rules.12–17It is important to acknowledge that the scope of the currentstudy was limited to the examination of chlamydia and/or gonor-rhea outcomes among asymptomatic women and heterosexualmen for several reasons. First, this article focused on this popu-lation for pragmatic reasons because sample size restrictionsprohibited the examination of other outcomes such as HIV andsyphilis, or other populations of interest such as men who havesex with men. Second, the targeted population accessing GCOwill be limited to those who are asymptomatic, as clients present-ing with symptoms will automatically be referred to an STI clinic.Derivation of Risk Estimation ToolThe outcomemeasuredwas diagnosiswith chlamydia and/orgonorrhea infection. We used χ2 tests to analyze categorical vari-ables and used Student t test to analyze continuous variables. Uni-variate logistic regression was used in the derivation populationto identify potential STI risk factors. To simplify risk score gen-eration and facilitate application in clinical and population-basedsettings, we categorized continuous variables (e.g., age and num-ber of sexual partners). We tested for interaction between sex andother risk factors. Predictors found to be significant in the uni-variate analyses were included in the final logistic regression modelusing backward elimination; predictors that remained in the modelhad P values less than 0.20. To be conservative, the final regres-sion model included only variables with P < 0.05 in at least oneof the imputed data sets.18 The risk factors in the final model wereused to construct the equation used for the clinical prediction rule.To aid use in screening decision making, we derived simplifiedrisk scores by multiplying the regression coefficients (β values)by 5 and rounding them to the nearest integers. Sum scores foreach visit were then derived by adding up the risk scores. Thesesum scores are a direct reflection of the probability of infection.19Performance MeasuresWe estimated the model's ability to discriminate betweenparticipants with or without infection as measured by the area un-der the receiver operating characteristic curve (AUC). An AUCvalue closer to a 100% shows that the model has excellent dis-criminative ability, whereas a value close to 50% indicates novalue.20,21 We performed 10-fold cross-validation techniques toestimate how the model will generalize to an independent popula-tion and correct for this optimism bias.22 We assessed calibrationperformance by calculating the Hosmer-Lemeshow goodness-of-fit statistic, which measures whether the predicted probabilityof infection corresponds with the observed probability. A well-calibrated model gives a corresponding P value greater than0.05.23We also studied the calibration of the simplified risk scoresby visually examining the prevalence of chlamydia and/or gonor-rhea infection in groups of the risk scores.24 We also examinedthe sensitivity (i.e., proportion of all cases identified) and fractionof patients who would need to be screened at different cutoffsof the risk scores. The benchmark set for a well-performing toolis one that identifies more than 90% of cases while screening60% or less of the population.25RESULTSDerivation PopulationF1Figure 1 is a flowchart showing the selection of clinic visitswhose data were used in this study. The chlamydia and/or gonor-rhea infection prevalence was 1.8% in the derivation population(n = 10,437). T1Table 1 shows the baseline distribution of candidatepredictors. The following were the demographic characteristicsof the majority of patient visits during the study period: male(67%), individuals between 30 and 39 years old (31%), and whiterace (74%). Individuals who reported consistent condom use dur-ing sexual contact comprised approximately 27% of clinic visits.Sexual contact with a CSW was documented in 13% of patientvisits (Table 1).Univariate predictors of chlamydia and/or gonorrhea infec-tion are shown in T2Table 2. The following predictors were not sig-nificantly associated with infection and were subsequently notincluded in the final logistic regression model: sex, condom use,sex with partners recruited online, IDU, sex with IDU, and sexwith CSW. We found no significant differences between the riskfactors and the outcome by sex. T3Table 3 shows the results of thefinal multivariable regression model used for developing the pre-diction rule. The model included age in years (categorized as14–19, 20–14, 25–29, 30–39, ≥40), race/ethnicity (white or non-white), number of sexual partners (0, 1–2, ≥3), previous chla-mydia diagnosis (yes or no), and previous gonorrhea diagnosis(yes or no).F2Figure 2 shows the ROCs for the chlamydia and/or gon-orrhea risk estimation model in the derivation and temporal valida-tion populations. The model demonstrated good discrimination inthe derivation population (AUC = 0.75; 95% confidence intervalFalasinnu et al.2 Sexually Transmitted Diseases • Volume 00, Number 00, Month 2014[CI], 0.72–0.80). Because internal validation indicates an upperlimit of the expected performance in new settings, the 10-foldcross-validation indicated the lack of evidence for overfitting(AUC = 0.74; 95% CI, 0.70–0.77). The model demonstratedstrong calibration in the derivation population, indicating goodfit; the Hosmer-Lemeshow χ2 statistic was 3.4 (8 df, P = 0.91).The coefficients yielded risk scores, with a minimum sum scoreof −2 and a maximum sum score of 26. To visualize the calibrationof the prediction rule, the total sample was divided into 6 groups,as shown inF3 Figure 3, which illustrates the observed proportionof chlamydia and/or gonorrhea infection as a function of thesum score derived from the final model. Higher sum scores werecorrelated with higher prevalence, further bolstering the goodcalibration indicated by the Hosmer-Lemeshow statistic.The simplified risk scores can be applied for selectivescreening decision making.T4 Table 4 shows the screening perfor-mance estimates at different cutoff levels of the sum scores. Toidentify all cases (i.e., 100% sensitivity), approximately 94% ofthe population would need to be screened at a sum score cutoffpoint of at least 1. However, by reducing the cutoff point of the riskscore to at least 6, only 68% of the population would need to bescreened to identify 91% of the cases, making this close to thebenchmark of screening of 60% or less while identifying morethan 90% of cases.Temporal Validation PopulationThe validation sample consisted of 14,956 clinic visits, ofwhich 2.2% were diagnosed with chlamydia and/or gonorrheainfection. There were no notable differences between the deriva-tion and validation populations; however, the temporal validationpopulation had lower prevalence of IDU, previous chlamydiadiagnosis, and previous gonorrhea diagnosis (Table 1). The modeldemonstrated acceptable discrimination in the temporal valida-tion population (AUC = 0.64; 95% CI, 0.61–0.67; Fig. 2). Themodel also showed good calibration upon validation (Hosmer-Lemeshow χ2 = 8.8, 8 df, P = 0.36). When categorized into thesame 6 risk categories as the derivation population, chlamydiaand/or gonorrhea infection prevalence ranged from 0.1% in thelowest-risk category to 16.1% in the highest-risk category (Fig. 3).When the simplified risk scores were considered in the temporalvalidation population, choosing the risk cutoff point of at least6 would identify 83% of cases while screening 68% of the popu-lation (Table 4).DISCUSSIONWe derived and validated a risk scoring tool for assessingthe risk of chlamydia and/or gonorrhea infection among asymp-tomatic women and heterosexual men accessing STI clinics inVancouver, BC, using predictors that are relatively easy to assessin a clinical encounter. The waning in discriminatory performanceof the model could be due to the difference in case-mix betweenthe 2 periods.26 Although the later time frame had a slightly higherprevalence of infection, individuals comprising this population re-ported lower proportions of the risk factors included in the finalmodel. As a result, the discrimination between cases and noncasesin the more homogeneous temporal validation population wasFIGURE 1. Study population selection.Chlamydia and Gonorrhea Risk AlgorithmSexually Transmitted Diseases • Volume 00, Number 00, Month 2014 3more difficult.26 In addition, there is also the possibility thatmissed risk factors could have impacted the discriminatory perfor-mance of the model in the temporal validation population. Futurestudies should consider reestimating the logistic regression modelto identify whether the inclusion of more risk factors will improvethe discriminatory performance of the model.27,28The aim of selective screening is to increase sensitivity(percentage of cases detected) and to increase efficiency (decreasethe percentage of the population to be screened). In the derivationpopulation, if screening were advised for people with a score ofat least 6, 91% of the cases would be detected by screening only68% of the population; the optimal threshold of the benchmarkof 90% sensitivity by screening 60% would be close to beingreached. However, the 32% reduction in the number of individualsthat need to be screened using the ≥6 risk score cutoff comparedwith screening the whole population shows that the efficiency ofscreening in population-based programs may be improved bytargeting screening in this way.23 Age less than 25 years is ascreening criterion used in the United States and Canada. 29,30TABLE 1. Population Characteristics of Derivation and TemporalValidation Populations*VariableDerivationPopulationTemporal ValidationPopulationn % n %Chlamydia/gonorrhea caseYes 184 1.8 331 2.2No 10,253 98.2 14,625 97.8SexFemale 3496 33.5 5341 35.7Male 6941 66.5 9615 64.3Age, y14–19 257 2.5 249 1.720–24 1962 18.8 2142 14.325–29 2651 25.4 4347 29.130–39 3181 30.5 4892 32.7≥40 2386 22.9 3327 22.2Race/EthnicityWhite 7732 74.1 10,402 69.6Nonwhite 2705 25.9 4554 30.4No. sexual partnersin previous 6 mo0 644 6.2 734 4.91–2 6857 65.7 9252 61.9≥3 2936 28.1 4971 33.2Condom use*Never 2362 22.6 3178 21.2Sometimes 5269 50.5 7630 51.0Always 2806 26.9 4148 27.7Sex with partnersrecruited onlineYes 417 4.0 1375 9.2No 10,020 96.0 13,581 90.8IDUYes 211 2.0 134 0.9No 10,226 98.0 14,822 99.1Sex with injectiondrug userYes 455 4.4 405 2.7No 9983 95.6 14,551 97.3Sex with CSWYes 1381 13.2 1840 12.3No 9057 86.8 13,116 87.7Previous chlamydiadiagnosisYes 1518 14.5 1728 11.6No 8919 85.5 13,228 88.4Previous gonorrheadiagnosisYes 619 5.9 441 2.9No 9819 94.1 14,515 97.1Total 10,437 100.0 14,956 100.0*Missing data among the predictor variables were handled using a mul-tiple imputation procedure with 5 resampling replications, which generatedan augmented databases with (5 * 10,437) 52,185 and (5 * 14,956) 74,780observations with complete data in the derivation and temporal validationpopulations, respectively. With imputed sample, we estimated baselinecharacteristics and developed prediction models. The average of all 5 im-puted samples is shown in this table.TABLE 2. Chlamydia and/or Gonorrhea Prevalence and Unad-justed ORs (Derivation Data Set)*VariableVancouver Clinics, 2000–2006 (N = 10,437)% OR (95% CI)SexFemale 2.1 1.31 (0.97–1.77)Male 1.6 ReferenceAge, y14–19 7.4 6.49 (3.58–11.75)20–24 2.8 2.30 (1.46–3.63)25–29 1.8 1.50 (0.94–2.38)30–39 1.1 0.88 (0.53–1.45)≥40 1.2 ReferenceRace/EthnicityWhite 1.2 ReferenceNonwhite 3.4 2.90 (2.16–3.89)No. sexual partnersin previous 6 mo0 0.5 Reference1–2 1.8 3.43 (1.10–10.73)≥3 2.0 3.92 (1.23–12.45)Condom useNever 1.6 ReferenceSometimes 2.0 1.22 (0.84–1.77)Always 1.4 0.88 (0.56–1.39)Sex with partnersrecruited onlineYes 1.2 0.67 (0.27–1.63)No 1.8 ReferenceIDUYes 1.6 0.89 (0.26–3.06)No 1.8 ReferenceSex with injectiondrug userYes 1.0 0.55 (0.20–1.54)No 1.8 ReferenceSex with CSWYes 1.2 0.65 (0.35–1.20)No 1.8 ReferencePrevious chlamydiadiagnosisYes 5.1 4.40 (3.27–5.93)No 1.2 ReferencePrevious gonorrheadiagnosisYes 2.1 1.21 (0.69–2.14)No 1.7 ReferenceTotal 1.8 —*Missing data among the predictor variables were handled using a mul-tiple imputation procedure with 5 resampling replications, which generatedan augmented databases with (5 * 10,437) 52,185 and (5 * 14,956) 74,780observations with complete data in the derivation and temporal validationpopulations, respectively. With imputed sample, we estimated baselinecharacteristics and developed prediction models. The average of all 5 im-puted samples is shown in this table.OR indicates odds ratio.Falasinnu et al.4 Sexually Transmitted Diseases • Volume 00, Number 00, Month 2014Using this criterion in the derivation population would requirescreening 21% of the population while identifying only 40% ofcases, a performance that falls short of the screening benchmarkeven after increasing the age cutoff to less than 30 years, indicatingthat the prediction rule could be useful in decision making in thissetting.The findings of this analysis were compared with a recentstudy that involved the derivation of a risk scoring tool for chla-mydia infection among sexual health clinic attendees in Sydney,Australia. 9 The models (which comprised demographics, sexualbehavior, and clinical symptoms) showed modest discrimination;the AUCwas 0.71 and 0.72 in heterosexual males and females, re-spectively. The screening efficiency benchmarkwas not reached inthe populations studied. A risk score cut-point of at least 15yielded a sensitivity of 90% and fraction screened of 87% in het-erosexual females, whereas a cut-point of at least 25 yielded asensitivity of 89% and fraction screened of 87% among heterosex-ual males. The AUC of the “Vancouver” risk estimation wasslightly higher compared with the AUCs of the aforementionedstudy. 9 The increase in this performance metric was also reflectedin the higher screening efficiency in the Vancouver risk scoringtool compared with the “Sydney” tool. This finding bolsters con-fidence in the predictive strength of the Vancouver algorithm, es-pecially because the algorithm excludes symptoms, which havebeen shown to be significantly associated with infection. 31The algorithm also has potential to inform screening deci-sions, especially in low STI prevalence settings. For example, toaid the scaling up of GCO, BC's Internet-based STI testing pro-gram, the algorithm could be adapted into a self-selection tool forfiltering GCO participants based on risk profile. Only participantswith sufficient risk score would be recommended to receive test-ing through GCO. 9 It is also anticipated that the prediction ruleFIGURE 2. Receiver operating curves for derivation and temporal validation populations.TABLE 3. Prediction Rule for Quantifying the Probability of CT/GCAQ2 InfectionVariable aOR (95% CI) β Scoring pointsIntercept — −6.0381 —Age, y14–19 5.25 (2.82–9.79) 1.6589 820–24 1.92 (1.19–3.10) 0.6544 325–29 1.20 (0.74–1.94) 0.1794 130–39 0.71 (0.42–1.18) −0.3471 −2≥40 Reference Reference ReferenceRace/EthnicityWhite Reference Reference ReferenceNonwhite 2.74 (2.03–3.70) 1.0093 5No. sexual partners in previous 6 mo0 Reference Reference Reference1–2 2.90 (0.97–8.75) 1.0565 5≥3 3.03 (0.99–9.30) 1.1000 6Previous chlamydia diagnosisYes 4.34 (3.19–5.89) 1.4574 7No Reference Reference ReferencePrevious gonorrhea diagnosisYes 1.27 (0.70–2.34) 0.2426 1No Reference Reference ReferenceaOR indicates adjusted odds ratio.Chlamydia and Gonorrhea Risk AlgorithmSexually Transmitted Diseases • Volume 00, Number 00, Month 2014 5could potentially facilitate decision making in traditional clinicalencounters where the algorithm could be used to display an alerton the computer screen to prompt clinicians to offer specific STItests to those at increased risk of infection. The prediction rulecould also be used to inform ongoing clinical recommendationsrelated to selective screening of STI clients and standardize STIFIGURE 3. Prevalence of chlamydia and/or gonorrhea within risk score categories.TABLE 4. Sensitivity and Specificity of Cutoff ScoresScore CutoffDerivation Population Temporal Validation PopulationSensitivity Specificity Fraction Screened PPV Sensitivity Specificity Fraction Screened PPV≥−2 100.0% 0.0% 100.0% 1.8% 100.0% 0.0% 100.0% 2.2%≥−1 100.0% 1.2% 98.8% 1.8% 100.0% 0.9% 99.1% 2.2%≥0 100.0% 1.3% 98.7% 1.8% 100.0% 0.9% 99.1% 2.2%≥1 100.0% 2.6% 97.4% 1.8% 99.9% 2.0% 98.1% 2.3%≥2 99.5% 3.7% 96.4% 1.8% 99.9% 2.7% 97.3% 2.3%≥3 99.5% 3.7% 96.4% 1.8% 99.9% 2.7% 97.3% 2.3%≥4 96.7% 16.7% 83.5% 2.0% 91.0% 15.7% 84.4% 2.4%≥5 95.8% 22.2% 78.1% 2.2% 87.2% 22.6% 77.7% 2.5%≥6 91.2% 32.7% 67.8% 2.4% 83.3% 32.3% 68.0% 2.7%≥7 84.9% 47.7% 52.8% 2.8% 73.4% 47.8% 52.7% 3.1%≥8 82.2% 53.0% 47.7% 3.0% 63.8% 54.4% 46.0% 3.1%≥9 72.2% 65.2% 35.4% 3.6% 53.5% 65.4% 35.0% 3.4%≥10 67.3% 70.6% 30.1% 3.9% 46.8% 71.2% 29.2% 3.6%≥11 62.5% 74.8% 25.8% 4.3% 42.5% 75.8% 24.6% 3.8%≥12 52.6% 81.4% 19.2% 4.8% 33.9% 83.6% 16.8% 4.5%≥13 47.7% 84.5% 16.1% 5.2% 30.4% 87.0% 13.4% 5.0%≥14 35.1% 91.1% 9.3% 6.6% 19.9% 92.0% 8.3% 5.3%≥15 32.4% 94.2% 6.3% 9.1% 14.5% 94.8% 5.4% 6.0%≥16 25.9% 95.9% 4.5% 10.1% 12.7% 96.2% 4.0% 7.0%≥17 21.2% 96.9% 3.4% 10.9% 10.3% 97.1% 3.0% 7.5%≥18 19.9% 97.2% 3.1% 11.4% 9.6% 97.4% 2.7% 7.9%≥19 14.3% 98.5% 1.8% 14.4% 6.1% 98.8% 1.3% 10.2%≥20 11.6% 99.0% 1.2% 16.9% 3.6% 99.5% 0.6% 14.1%≥21 7.3% 99.5% 0.6% 20.9% 1.9% 99.8% 0.3% 16.1%≥22 2.8% 99.9% 0.2% 28.0% 0.6% 100.0% 0.1% 23.8%≥23 2.8% 99.9% 0.1% 33.8% 0.6% 100.0% 0.1% 24.4%≥24 2.8% 99.9% 0.1% 33.8% 0.6% 100.0% 0.1% 24.4%≥25 2.8% 99.9% 0.1% 33.8% 0.6% 100.0% 0.1% 24.4%≥26 0.7% 100.0% 0.0% 30.0% 0.3% 100.0% 0.0% 33.3%PPVAQ3 indicates predictive positive value.Falasinnu et al.6 Sexually Transmitted Diseases • Volume 00, Number 00, Month 2014testing at the clinics in BC, potentially enabling targeted testing,thereby reducing the unnecessary testing of those without theinfection and saving costs. The algorithm could be helpful inprioritizing or triaging patients. In sexual health clinics, triageservices require patients to fill out risk assessment question-naires in the waiting room before receiving services. 2,5,32 The riskestimation algorithm could be adapted into a decision-making toolthat prioritizes patients by risk scores. This may enable cliniciansto effectively determine whether triaging those with low-riskscores to receive “express” services (e.g., providing self-collectedspecimens) or referring others to more comprehensive services(e.g., complete physical examinations). 8 This process may helpdecrease wait times, improve clinicalworkflow, and reduce unnec-essary clinician face time.Our study has some limitations. The data from which themodel was derived came from STI clinics and this could limit gen-eralizability to other settings such as reproductive health clinics orgeneral practice settings. The generalizability of the model in addi-tional STI clinics outside Vancouver, BC, will be the subject of aforthcoming paper. One assumption the algorithm makes is thatof a fixed epidemic; however, this assumption would need to bereevaluated over time to reflect the evolution of patient risk pro-files, a possibility that could be facilitated by the adoption of elec-tronic medical records. Also, although we examined chlamydiaand/or gonorrhea infection as a composite outcome, we conducteda subanalysis and found that there was no difference in the perfor-mance of the prediction rule for detecting gonorrheal infection(AUC = 0.73) and chlamydia infection (AUC = 0.75) only. Thisfinding gave us more confidence in combining both outcomes.In conclusion, we derived a pragmatic risk scoring toolfrom a model that included a diverse patient population. Asfunding available for sexual health services decrease (or in somecases, remain stagnant) and STI rates increase, public health pro-grams are in need of novel strategies to maximize service effi-ciency. Future research is needed to determine whether theadoption of risk estimation tools such as the one developed herewill result in economic savings and long-term impact on resourceallocation (including health human services and clinic operations).REFERENCES1. Rietmeijer CA, Mettenbrink C. Why we should save our STD clinics.Sex Transm Dis 2010;37:591.2. Fairley CK, Williams H, Lee DM, et al. A plea for more research onaccess to sexual health services. Int J STD AIDS 2007;18:75–76.3. Golden MR, Kerndt PR. Improving clinical operations: Can we andshould we save our STD clinics? Sex Transm Dis 2010;37:264–265.4. Hottes TS, Farrell J, Bondyra M, et al. Internet-based HIVandsexually transmitted infection testing in British Columbia, Canada:Opinions and expectations of prospective clients. J Med Internet Res2012;14:e41.5. Shamos SJ, Mettenbrink CJ, Subiadur JA, et al. Evaluation of atesting-only “express” visit option to enhance efficiency in a busy STIclinic. Sex Transm Dis 2008;35:336–340.6. Falasinnu T, Gustafson P, Gilbert M, et al. Risk prediction in sexualhealth contexts: Protocol. JMIR Res Protoc 2013;2:e57.7. Geisler WM, Chow JM, Schachter J, et al. Pelvic examinationfindings and Chlamydia trachomatis infection in asymptomatic youngwomen screenedwith a nucleic acid amplification test. Sex TransmDis2007;34:335–338.8. van den Broek IV, Brouwers EE, Gotz HM, et al. Systematic selectionof screening participants by risk score in a chlamydia screeningprogramme is feasible and effective. Sex Transm Infect2012;88:205–211.9. Wand H, Guy R, Donovan B, et al. Developing and validating a riskscoring tool for chlamydia infection among sexual health clinicattendees in Australia: A simple algorithm to identify those at high riskof chlamydia infection. BMJ Open 2011;1:e000005.10. Falasinnu T, Gustafson P, Hottes TS, et al. A critical appraisal of riskmodels for predicting sexually transmitted infections. Sex Transm Dis2014;41:321–330.11. Mahony JB, Luinstra KE, Tyndall M, et al. Multiplex PCR fordetection of Chlamydia trachomatis and Neisseria gonorrhoeae ingenitourinary specimens. J Clin Microbiol 1995;33:3049–3053.12. Heymans MW, van Buuren S, Knol DL, et al. Variable selection undermultiple imputation using the bootstrap in a prognostic study.BMC Med Res Methodol 2007;7:33.13. Janssen KJ, Donders AR, Harrell FE Jr, et al. Missing covariate data inmedical research: To impute is better than to ignore. J Clin Epidemiol2010;63:721–727.14. Janssen KJ, Vergouwe Y, Donders AR, et al. Dealing with missingpredictor values when applying clinical prediction models. Clin Chem2009;55:994–1001.15. Vergouw D, Heymans MW, Peat GM, et al. The search for stableprognostic models in multiple imputed data sets. BMC Med ResMethodol 2010;10:81-2288-10-81.16. Survey Research Center, Institute for Social Research, University ofMichigan. IVEware: Imputation and Variance Estimation software.Available at: http://www.isr.umich.edu/src/smp/ive/. Updated 2013.Accessed August 7, 2013.17. He Y, Raghunathan T. On the performance of sequential regressionmultiple imputation methods with non normal error distributions.Commun Stat Simul Comput 2009;38:856–883.18. Vergouwe Y, Royston P, Moons KG, et al. Development and validationof a prediction model with missing predictor data: A practicalapproach. J Clin Epidemiol 2010;63:205–214.19. Gotz HM, van Bergen JE, Veldhuijzen IK, et al. A prediction rule forselective screening of Chlamydia trachomatis infection. Sex TransmInfect 2005;81:24–30.20. Justice AC, Covinsky KE, Berlin JA. Assessing the generalizability ofprognostic information. Ann Intern Med 1999;130:515–524.21. Steyerberg EW. Clinical Prediction Models: A Practical Approach toDevelopment, Validation, and Updating. New York: Springer, 2009.22. Vickers AJ, Cronin AM. Traditional statistical methods forevaluating prediction models are uninformative as to clinical value:Towards a decision analytic framework. Semin Oncol 2010;37:31–38.23. Gotz HM, Veldhuijzen IK, Habbema JD, et al. Prediction ofChlamydia trachomatis infection: Application of a scoring rule to otherpopulations. Sex Transm Dis 2006;33:374–380.24. Haukoos JS, LyonsMS, Lindsell CJ, et al. Derivation and validation ofthe Denver human immunodeficiency virus (HIV) risk score fortargeted HIV screening. Am J Epidemiol 2012;175:838–846.25. La Montagne DS, Patrick LE, Fine DN, et al. Region X InfertilityPrevention Project. Re-evaluating selective screening criteria forchlamydial infection among women in the U S Pacific Northwest.Sex Transm Dis 2004;31:283–289.26. Toll DB, Janssen KJ, Vergouwe Y, et al. Validation, updating andimpact of clinical prediction rules: A review. J Clin Epidemiol 2008;61:1085–1094.27. Moons KG, Kengne AP, Grobbee DE, et al. Risk prediction models: II.External validation, model updating, and impact assessment. Heart2012;98:691–698.28. Rosser BR, Miner MH, BocktingWO, et al. HIV risk and the Internet:Results of the Men's INTernet Sex (MINTS) study. Aids Behav 2009;13:746–756. Available at: http://ovidsp.ovid.com/ovidweb.cgi?T=JS&CSC=Y&NEWS=N&PAGE=fulltext&=medl&AN=18512143.29. Meyers D, Wolff T, Gregory K, et al. USPSTF recommendations forSTI screening. Am Fam Physician 2008; 77:819–824.30. Public Health Agency of Canada. Canadian Guidelines On SexuallyTransmitted Infections—Updated January 2010. Available at: http://www.phac-aspc.gc.ca/std-mts/sti-its/cgsti-ldcits/index-eng.php.Updated 2011. Accessed July 22, 2013.31. Xu F, Stoner BP, Taylor SN, et al. “Testing-only” visits: An assessmentof missed diagnoses in clients attending sexually transmitted diseaseclinics. Sex Transm Dis 2013;40:64–69.32. Martin L, Knight V, Ryder N, et al. Client feedback and satisfactionwith an express sexually transmissible infection screening service at aninner-city sexual health center. Sex Transm Dis 2013;40:70–74.Chlamydia and Gonorrhea Risk AlgorithmSexually Transmitted Diseases • Volume 00, Number 00, Month 2014 7AUTHOR QUERIESAUTHOR PLEASE ANSWER ALL QUERIESAQ1 = Please check if the captured correspondence address is accurate.AQ2 = Please define CT/GC in the legend.AQ3 = Please check if PPV was defined correctly.END OFAUTHOR QUERIES


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items