UBC Faculty Research and Publications

Successful methodology for large-scale surveillance of severe events following influenza vaccination… Bettinger, Julie A.; Rouleau, Isabelle; Gariépy, Marie-Claude; Bowie, William R.; Valiquette, Louis; Vanderkooi, Otto G.; Kellner, James D.; Coleman, Brenda; McNeil, Shelly; McCarthy, Anne; De Serres, Gaston 2015

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-Bettinger_J_et_al_Successful_methodology.pdf [ 305.39kB ]
JSON: 52383-1.0228080.json
JSON-LD: 52383-1.0228080-ld.json
RDF/XML (Pretty): 52383-1.0228080-rdf.xml
RDF/JSON: 52383-1.0228080-rdf.json
Turtle: 52383-1.0228080-turtle.txt
N-Triples: 52383-1.0228080-rdf-ntriples.txt
Original Record: 52383-1.0228080-source.json
Full Text

Full Text

1www.eurosurveillance.orgSurveillance and outbreak reportsSuccessful methodology for large-scale surveillance of severe events following influenza vaccination in Canada, 2011 and 2012J A Bettinger (jbettinger@cfri.ca)1, I Rouleau2, M C Gariépy2, W R Bowie3, L Valiquette4, O G Vanderkooi5, J D Kellner5, B L Coleman6, S A McNeil7, A McCarthy8, G De Serres2,9, On behalf of the Public Health Agency of Canada/Canadian Institutes for Health Research Influenza Research Network1. Vaccine Evaluation Center, BC Children’s Hospital, University of British Columbia, Vancouver, British Columbia, Canada2. Centre de Recherche du CHU de Québec, Laval University, Canada3. Vancouver General Hospital, University of British Columbia, Vancouver, British Columbia, Canada4. Centre Hospitalier Universitaire de Sherbrooke, Sherbrooke, Quebec, Canada5. Alberta Children’s Hospital, University of Calgary, Calgary, Alberta, Canada6. Mount Sinai Hospital, University of Toronto, Toronto, Ontario, Canada7. Canadian Center for Vaccinology, IWK Health Centre, Queen Elizabeth II Health Sciences Centre and Dalhousie University, Halifax, Nova Scotia, Canada8. Ottawa Hospital, Ottawa, Ontario, Canada9. Institut National de Santé Publique du Quebec, Quebec City, Quebec, CanadaCitation style for this article: Bettinger JA, Rouleau I, Gariépy MC, Bowie WR, Valiquette L, Vanderkooi OG, Kellner JD, Coleman BL, McNeil SA, McCarthy A, De Serres G, On behalf of the Public Health Agency of Canada/Canadian Institutes for Health Research Influenza Research Network. Successful methodology for large-scale surveillance of severe events following influenza vaccination in Canada, 2011 and 2012. Euro Surveill. 2015;20(29):pii=21189. Available online: http://www.eurosurveillance.org/ViewArticle.aspx?ArticleId=21189  Article submitted on 21 July 2014 / published on 23 July 2015In 2011 and 2012, a nationwide Canadian vaccine safety surveillance network rapidly collected safety data from healthcare workers (HCW) during the first weeks of the annual influenza vaccination campaign. This network provided the first available post-marketing safety data on seasonal influenza vaccines with information on background rates as a comparator. In 2012, these data were used to investigate a possible safety concern regarding a particular vaccine. An online question-naire was provided to participating HCW two weeks before the annual influenza vaccination campaign for controls, and eight days after influenza vaccination for vaccinees. Control and vaccinees were requested to report health events occurring in the seven days prior to receiving the questionnaire. Control data were used to calculate background rates. HCW reporting a severe event were followed-up by telephone within 48 hours of the online report to validate the report and check on their health status. More than 22,000 vaccinated HCW were enrolled and surveyed over two seasons and > 90% reported no severe event following vaccina-tion. Validated severe event rates were similar in vac-cinated HCW and unvaccinated HCW (2.2% vs 2.3%; p < 0.70). The questionnaire was accurately completed for most reported symptoms, matched the validated report and was able to detect events of interest. Prior to the safety concern, the implicated vaccine was in use at one centre. Reassuring safety data were provided to public health authorities 48 hours after the vaccine was temporarily suspended. Data from this and similar networks can be used for rapid evaluation of vaccine safety and for safety assessment as required by the European Medicines Agency in 2015.IntroductionInfluenza vaccines are modified yearly to include the influenza viral strains most likely to circulate during the next influenza season. Starting in 2015, European regulatory requirements to evaluate the safety and immunogenicity of seasonal influenza vaccines in small scale clinical trials will be withdrawn [1]. Such trials had insufficient power to adequately evaluate safety concerns arising from annual formulation changes (e.g. adverse events occurring at a rate of 1–2%). These clin-ical trials are to be replaced by enhanced, preferably active, safety monitoring and vaccine effectiveness assessments [2]. Recognising the need for timely information support-ing the seasonal vaccines’ safety profiles early in the annual immunisation campaign, a sentinel network was established in Canada in 2009 [3] to conduct online safety monitoring. The goals of the online sur-veillance are to detect any safety signals and provide an estimate of severe events reported in vaccinated and non-vaccinated individuals. As part of the sentinel network, methodology for online, active, safety monitoring was further refined and tested and is described here. This study of the network surveillance aimed to assess health events reported following vaccination, by healthcare workers 2 www.eurosurveillance.orgTable 1Characteristics of Canadian health care workers enrolled in 2011 and 2012, who responded to an online questionnaire and randomly selected non-responders contacted for validation in 2012, Canada, 2011–2012 (n=43,776 healthcare workers)2011/12 influenza season 2012/13 influenza seasonControlsN (%)VaccineesN (%)P-valueaControlsN (%)Control initial non-responderN (%)P-valuebVaccineesN (%)Vaccinee initial non-responder N (%)P-valueEnrollment and participationEnrolled 12,238 (100) 10,070 (100) – 9,458 (100) NA – 12,010 (100) NA –Response (rate) 1,616 (13.2) 7,496 (74.4) – 2,479 (26.2) 921 (9.7) – 7,667 (63.8) 994 (8.3) –SexFemale 1,239 (76.7) 5,462 (72.9) 0.002 1,849 (74.6) 690 (74.9) 0.84 5,634 (73.5) 749 (75.4) 0.28c; 0.21dAge group, in years<30 270 (16.7) 1,786 (23.8)<0.001468 (18.9) 197 (21.4)0.301,882 (24.5) 218 (21.9)<0.001c; <0.001d30–39 351 (21.7) 1,735 (23.2) 609 (24.6) 201 (21.8) 1,830 (23.9) 199 (20)40–49 345 (21.4) 1,656 (22.1) 552 (22.3) 200 (21.7) 1,668 (21.8) 210 (21.1)50–59 479 (29.6) 1,683 (22.5) 615 (24.8) 228 (24.8) 1,688 (22.0) 260 (26.2)≥60 167 (10.3) 632 (8.4) 235 (9.5) 95 (10.3) 599 (7.8) 107 (10.8)OccupationPhysician 210 (13.0) 1,049 (14.0)<0.001367 (14.8) 101 (11.0)<0.0011,130 (14.7) 88 (8.9)<0.001c; <0.001dNurse/assistant nurse 292 (18.1) 1,498 (20.0) 439 (17.7) 231 (25.1) 1,537 (20) 251 (25.3)Patient care assistant 10 (0.6) 123 (1.6) 19 (0.8) 28 (3.0) 102 (1.3) 47 (4.7)Medical technologist 117 (7.3) 543 (7.2) 178 (7.2) 81 (8.8) 537 (7) 81 (8.1)Technician/other health professional 293 (18.2) 1,126 (15.0) 405 (16.3) 110 (11.9) 1,192 (15.5) 121 (12.2)Housekeeping, logistics/food service50 (3.1) 357 (4.8) 45 (1.8) 40 (4.3) 211 (2.8) 70 (7)Administrative/office 348 (21.6) 1,281 (17.1) 486 (19.6) 180 (19.5) 1,582 (20.6) 211 (21.2)Trainee/student 51 (3.2) 432 (5.8) 139 (5.6) 39 (4.2) 560 (7.3) 56 (5.6)Research 148 (9.2) 456 (6.1) 258 (10.4) 49 (5.3) 527 (6.9) 27 (2.7)Volunteer 0 (0) 0 (0) 71 (2.9) 24 (2.6) 244 (3.2) 38 (3.8)Other 94 (5.8) 353 (4.7) 71 (2.9) 38 (4.1) 45 (0.6) 4 (0.4)NA: not applicable.a  P-value for difference in characteristics between controls and vaccinees who respectively responded to the online questionnaire in 2011.b  P-value for difference in characteristics between controls who responded to the online questionnaire and controls who initially did not respond to the questionnaire (controls initial non-responders) in 2012.c  P-value for difference in characteristics between controls and vaccinees who respectively responded to the online questionnaire in 2012. d  P-value for difference in characteristics between vaccinees who responded to the online questionnaire and the vaccinees who initially did not respond to the questionnaire (vaccinees initial non-responders) in 2012.3www.eurosurveillance.orgTable 2 Number and rate of health events reported by healthcare workers preceding (controls) or following (vaccinees) influenza vaccination in 2011 and 2012 and health events reported by initial non-responders among controls and vaccinees in 2012, Canada, 2011–2012 (n=1,922 health events)Type of event reported2011 2012 Both years 2012Controls(N=1,616)n (%)Vaccinees(N=7,496)n (%)Controls(N=2,479)n (%)Vaccinees(N=7,667)n (%)Controls(N=4,095)n (%)Vaccinees(N=15,163)n (%)P-valueaControl initial non-responder(N=921)n (%)P-valuebVaccinee initial non-responder(N=994)n (%)P-valuecAny event 164 (10.1) 696 (9.3) 232 (9.4) 692 (9.0) 396 (9.7) 1,388 (9.2) 0.31 77 (8.4) 0.38 61 (6.1) 0.002 Severe eventd 52 (3.2) 155 (2.1) 97 (3.9) 233 (3.0) 149 (3.6) 388 (2.6) <0.001 36 (3.9) 0.84 27 (2.7) 0.62 Validated evente 25 (1.5) 127 (1.7) 69 (2.7) 206 (2.7) 94 (2.3) 333 (2.2) 0.70 NA – NA –Local reaction NA 102 (1.4) NA 99 (1.3) NA 201 (1.3) – NA – 17 (1.7) 0.30 Severe eventd NA 14 (0.2) NA 17 (0.2) NA 31 (0.2) – NA – 0 (0.0) 0.61 Validated evente NA 2 (0.03) NA 5 (0.1) NA 7 (0.0) – NA – NA –Systemic symptoms 35 (2.2) 239 (3.2) 73 (2.9) 336 (4.4) 108 (2.6) 575 (3.8) <0.001 28 (3.0) 0.91 28 (2.8) 0.02 Severe eventd 24 (1.5) 95 (1.3) 52 (2.1) 172 (2.2) 76 (1.9) 267 (1.8) 0.68 14 (1.5) 0.33 17 (1.7) 0.35 Validated evente 0 (0.0) 30 (0.4) 11 (0.4) 53 (0.7) 11 (0.3) 83 (0.5) 0.02 NA – NA –Allergy-like events 1 (0.06) 7 (0.09) 7 (0.3) 4 (0.05) 8 (0.2) 11 (0.1) 0.03 4 (0.4) 0.50 1 (0.1) 0.46 Severe eventd 0 (0.0) 4 (0.05) 4 (0.2) 2 (0.03) 4 (0.1) 6 (0.0) 0.15 2 (0.2) 0.67 1 (0.1) 0.31 Validated evente 0 (0.0) 1 (0.01) 0 (0.0) 1 (0.01) 1 (0.0) 2 (0.0) 0.61 NA – NA –Respiratory symptoms 31 (1.9) 124 (1.6) 63 (2.5) 131 (1.7) 94 (2.3) 255 (1.7) 0.01 29 (3.2) 0.34 17 (1.7) >0.99 Severe eventd 22 (1.4) 74 (1.0) 43 (1.7) 84 (1.1) 65 (1.6) 158 (1.0) 0.004 13 (1.4) 0.65 15 (1.5) 0.26 Validated evente 16 (1.0) 58 (0.8) 39 (1.6) 73 (1.0) 52 (1.3) 131 (0.9) 0.02 NA – NA –GI symptoms 11 (0.7) 67 (0.9) 29 (1.2) 97 (1.3) 40 (1.0) 164 (1.1) 0.56 12 (1.3) 0.73 6 (0.6) 0.09 Severe eventd 10 (0.6) 39 (0.5) 22 (0.9) 71 (0.9) 32 (0.8) 110 (0.7) 0.71 10 (1.1) 0.56 5 (0.5) 0.21 Validated evente 6 (0.4) 20 (0.3) 7 (0.3) 34 (0.4) 13 (0.3) 54 (0.4) 0.71 NA – NA –ORSf 15 (0.9) 191 (2.5) 87 (3.5) 163 (2.1) 102 (2.5) 354 (2.3) 0.56 22 (2.4) 0.10 13 (1.3) 0.09 Severe eventd 3 (0.2) 21 (0.3) 13 (0.5) 21 (0.3) 16 (0.4) 42 (0.3) 0.24 5 (0.5) >0.99 3 (0.3) 0.75 Validated evente 0 (0.0) 2 (0.03) 0 6 (0.08) 0 (0.0) 8 (0.1) – NA – NA –Paraesthesiaf 4 (0.3) 165 (2.2) 49 (2.0) 85 (1.1) 53 (1.3) 250 (1.6) 0.11 5 (0.5) 0.002 6 (0.6) 0.18 Severe eventd 0.7 (0.04) 9 (0.1) 9 (0.4) 7 (0.09) 10 (0.2) 16 (0.1) 0.03 2 (0.2) 0.74 3 (0.3) 0.10 Validated evente 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) 0 (0.0) – NA – NA –Other 7 (0.4) 54 (0.7) 31 (1.3) 101 (1.3) 38 (0.9) 155 (1.0) 0.59 20 (2.2) 0.05 11 (1.1) 0.66 Severe eventd 3 (0.2) 16 (0.2) 21 (0.8) 38 (0.5) 24 (0.6) 54 (0.4) 0.04 12 (1.3) 0.24 8 (0.8) 0.24 Validated evente 3 (0.2) 14 (0.2) 10 (0.4) 34 (0.4) 13 (0.3) 48 (0.3) 0.99 NA – NA –GI: gastrointestinal; NA: not applicable; ORS: oculorespiratory syndrome.a  P-value for difference in health event rate between controls and vaccinees in both years (2011, 2012). Significance ≤ 0.002 adjusted for multiple comparisons using Bonferroni’s correction.b P-value for difference in health event rate between controls who responded to the online questionnaire and controls who initially did not  respond (controls initial non-responders) in 2012. Significance ≤ 0.002 adjusted for multiple comparisons using Bonferroni’s correction.c  P-value for difference in health event rate between vaccinees who responded to the online questionnaire and vaccinees who initially did not respond (vaccinee initial non-responders) in 2012. Significance ≤ 0.002 adjusted for multiple comparisons using Bonferroni’s correction.d  Severe event is defined as a health event preventing daily activities or causing work absenteeism or requiring a medical consultation, or any combination of these effects.e  Validated events are  self-reported severe events in the online questionnaire, which remained the primary diagnosis after a nurse follow-up.f  Affected per 24 hours. The number of ORS and paraesthesia events for 2011 controls were divided by seven to adjust for the difference in reporting period for these controls. The reporting period for the 2011 controls was seven days, compared with 24 hours for controls in 2012 as well as for vaccinees in 2011 and 2012.4 www.eurosurveillance.org(HCW) participating in an online questionnaire. To establish background rates for health events, non-vaccinated HCW were also recruited to respond to the questionnaire. Telephone follow-up of participants reporting severe health events allowed estimation of the validity of such self-reported events. The rep-resentativeness of health events reported by online responders was also assessed by comparing the rates of health events in participants who responded to the online questionnaire to those who did not. The study was conducted during the two immunisation seasons of 2011 and 2012, whereby in 2012, prior to the tem-porary suspension of a seasonal influenza vaccine [4], some data were collected. The brief, voluntary suspen-sion of the vaccine offered a valuable opportunity to assess the capacity of the Canadian network’s ability to detect any signal of severe events post-vaccination and to rapidly provide safety data to public health deci-sion makers. MethodsOnline surveillance systemHCW who received the influenza vaccine in 2011 or 2012 were recruited to participate in an online sur-vey from seven and eight Canadian acute care hospi-tal sites respectively, in Alberta (2012 only), British Columbia, Nova Scotia, Ontario, and Quebec. The HCW were invited to enrol in the study when presenting for vaccination at a participating hospital, and provided their email address, telephone number(s) and informed consent. Enrolled vaccinated HCW were sent an email eight days after vaccination with a link to an online health event questionnaire. Vaccinee non-responders were sent a reminder email three days later. Two weeks before the start of the 2011 and 2012 vac-cination campaigns respectively at seven of the eight sites, HCW immunised in the previous year were invited to serve as a control group to establish the background rates for health events. Conducting the control survey before the start of influenza vaccination allowed for compliance with national recommendations for all HCW to receive the influenza vaccine and provided a compa-rable control group for vaccinees. Controls were sent an email with an embedded link to the online surveil-lance questionnaire which remained active until the day before the start of their institution’s influenza vaccina-tion programme (SimpleSurvey v2.17.0, OutSideSoft Solutions inc., Saint-Jean-sur-Richelieu, Quebec). Non-responders in the control group were sent a reminder email three days after the initial email link was sent. Participants were identified by a unique study code and email addresses were not linked with the ques-tionnaire responses. The study was approved by the research ethics boards at each site.The online questionnaires collected information on demographics (i.e. age, sex, occupation), past influ-enza vaccination history and occurrence of health events of interest. Health events occurring in the seven days before receiving the questionnaire link were docu-mented by broad categories: local injection site reac-tions (vaccinated HCW only), systemic symptoms (fever as temperature ≥38.5°C, fatigue, myalgia), respiratory symptoms suggestive of allergy-like events, bronchitis, cold, gastrointestinal symptoms (diarrhoea, nausea, vomiting), influenza, pharyngitis, pneumonia, sinusi-tis, tonsillitis and any other health event. Symptoms of oculorespiratory syndrome (ORS) and numbness (anaesthesia/paraesthesia) were also solicited [5-7]. ORS was defined according to the National Advisory Committee on Immunization definition [7]. All events were considered severe if they prevented daily activi-ties, resulted in work absenteeism, or required a medi-cal consultation.Observed health events and capacity for signal investigationFor each study year, rates of health events reported following vaccination were compared with those observed among controls. Symptoms of ORS or par-aesthesia beginning within a seven-day observation period for controls in 2011 or during the previous 24 hours for 2012 controls or 24 hours after vaccination for 2011 and 2012 vaccinees were reported (the word-ing for these questions was changed in 2012 to make the time period in controls and vaccinees comparable). Before comparison, the number of ORS and paraes-thesia events in 2011 controls were divided by seven to adjust for the difference in reporting period in 2011 controls. Characteristics and events were compared using chi-squared and Fischer exact tests.Following a safety signal issued by Italian authorities and a request from the Public Health Agency of Canada, Novartis Vaccines temporarily suspended Canadian dis-tribution of Agriflu on 26 October 2012 [8,9]. Clumping of virus-like particles in the vaccine prompted concerns about the potential for an increase in ORS or allergy-like reactions. The vaccine had already been in use at some of our sites; therefore, we had safety infor-mation on the implicated product prior to its tempo-rary suspension. In response to this safety signal, our network compared the event rates observed follow-ing vaccination with Agriflu (the signal vaccine) with those observed in controls and after receipt of other seasonal vaccines (Vaxigrip and Fluviral). Individual level data on the vaccines used for HCW vaccination were not available, although 7/8 centres vaccinated HCW with a single product (2 used Fluviral, 4 Vaxigrip and 1 Agriflu exclusively). Only one centre used both Agriflu and Vaxigrip, in unknown proportions and was excluded from this sub-analysis.Validity of self-reported events by healthcare personnelParticipants who reported any severe health event (i.e. prevented daily activities/work or required a medi-cal consultation) were contacted within 48 hours by a nurse trained in adverse events following vaccination 5www.eurosurveillance.org(AEFI) reporting who verified and documented the health event. Participants were allowed to enter several health events of interest occurring during the week, but nurses were required to choose a single primary event (i.e. the main complaint) based on their clinical judgement after speaking with the participant for each individual case report. Respiratory or gastrointestinal symptoms were considered the primary event when reported in conjunction with systemic symptoms. We excluded acute health events that had an onset before vaccination (for vaccinees) or > 1 week before the sur-vey (for controls), as well as events that did not meet the reporting criteria for a severe event, and scheduled medical visits. All events were reviewed and validated centrally by members of the research team (IR, MCG).Representativeness of events reported by respondersIn 2012, 10% of study participants (vaccinees and con-trols) who did not complete the online questionnaire after the reminder email (non-responders) were ran-domly selected and contacted by telephone five to 10 days after the reminder email was sent. A minimum of five attempts to contact each non-responder was made on different days and at different times before another non-responder was selected. We compared character-istics and rates of events observed between online responders and non-responders using Bonferonni’s correction for the chi-squared and Fischer exact tests.ResultsStudy participantsOver the two seasons, 22,080 vaccinated HCW enrolled in the surveillance network (Table 1). Overall, 15,163 (68.7%) responded to the online questionnaire sent eight days following vaccination, although the response rate was statistically higher in 2011 compared with the following year (74.4% vs 63.8%; p < 0.001) (Table 1). The characteristics of HCW who responded Any severe eventLocal reaction ORSFever ≥ 38.5°CFatigue/myalgiaAllergy-like eventsRespiratory symptomsGI symptomsProportion of severe events4.543.532.521.510.50Signal vaccineOther seasonal vaccinesControlsFigureRates of severe health events reported in healthcare workers vaccinated with one seasonal vaccine (n=1,084) compared with respective rates in those vaccinated with other seasonal vaccines (n=6,360) and controls (n=3,400), Canada, 2012GI: gastrointestinal; ORS: oculorespiratory syndrome.6 www.eurosurveillance.orgto the online surveillance remained stable over the two study years: about three quarters of participants were women, two thirds were between 30 and 59 years of age and the majority (10,397/15,163; 68.5%) were involved in patient care. Most (14,329/15,163; 94.5%) respondents had been vaccinated against influenza in the past and 68.8% (n=10,432/15,163) reported receiv-ing the vaccine annually in the last three years. For the control questionnaire, 12,238 HCW immunised during the previous season were contacted in 2011 and 9,458 in 2012, with response rates of 13.2% and 26.2%, respectively (p < 0.001) (Table 1). Demographic characteristics of controls and vac-cinees are shown in Table 1. In both years, respond-ents younger than 30 years of age were slightly more represented in the vaccinated group than in the con-trols, and there were minor differences with regard to profession/occupation. Observed health events and capacity for signal investigationOver the two study years, 1,388 (9.2%) health events were self-reported by vaccinated HCW compared with 396 (9.7%) health events by controls (p = 0.31) (Table 2). Among vaccinated HCW, 388 reported events (2.6%) were severe enough to result in work absentee-ism, prevent daily activities or to require a medical con-sultation, compared with 149 (3.6%; p < 0.001) reported among controls. Overall, 43.9% (609/1,388) of events self-reported by vaccinated HCW included respiratory or ORS symptoms, which was similar to the proportion reported by controls (49.5%; 196/396). However, sys-temic symptoms were more frequently reported by vac-cinated HCW (41.4%; 575/1,388) than controls (27.3%; 108/396) respectively (p < 0.001; Table 2).The hospitals surveyed in this study conducted their yearly vaccinations campaigns earlier than most juris-dictions in Canada and HCW at one of these hospitals were vaccinated with the vaccine implicated in the safety concern before it was temporarily suspended. This allowed a comparative safety review of the sig-nal vaccine severe event rates with the other seasonal influenza vaccines and the background rates observed in controls within 48 hours of the vaccine suspension. A sub-analysis of the 2012 dataset was conducted including the vaccinees who responded to the ques-tionnaire (n=7,667), the vaccinated initial online non-responders who subsequently provided information (n=994), but excluding the one site with mixed vaccine use (n=1,217). This confirmed the interim findings. A total of 1,084 of the 7,444 (15%) vaccinees received the signal vaccine. The rate of self-reported severe events among HCW vaccinated with the signal vac-cine was 2.9% (31/1,084), which was similar to 2.4% (151/6,360; p = 0.40) in HCW vaccinated at institutions using other seasonal vaccines and to all controls at 3.9% (133/3,400; p = 0.11). The clinical nature of severe health events reported by HCW vaccinated with the signal vaccine was similar to those reported after other seasonal vaccines (Figure).Validity of self-reported eventsOver the two study seasons, nurses were able to com-plete follow-up calls with 93% (500/537) of participants reporting severe events online (Table 3). This resulted in 90% (134/149) of controls and 94% (366/388) of vac-cinated HCW being followed-up (p = 0.09). Following the nurse interviews, 30% (40/134) of controls and 9% (33/366) of vaccinees reporting severe events were excluded, leaving a total of 427 participants with eli-gible severe health events (94 controls and 333 HCW). Reasons for exclusions were that these events (i) started > 1 week before the survey (for controls) or prior to vaccination (for vaccinees), (ii) did not prevent daily activities/work or require a medical consultation or (iii) were previously scheduled medical visits. In both years, the proportion of events that were excluded was significantly higher among controls than that observed among vaccinated individuals (30% vs 9%; p < 0.001). Participants who were excluded did not vary according to the type of event or clinical presentation, with the notable exception of paraesthesia, which was a pre-existing condition in all controls not considered.The accuracy and validity of the online reported severe health events are shown in Table 3. Among 427 par-ticipants reporting eligible severe health events, 45% (n=193) had respiratory symptoms. For 79% (n=153) of these, respiratory symptoms remained the primary diagnosis after talking to the nurse (i.e. validated event). Gastrointestinal symptoms were reported by 28% (121/427) of participants with eligible severe health events, and of those reports, 54% (65/121) remained as the primary diagnosis. Eligible severe sys-temic symptoms were frequently reported (59% in con-trols and 71% in vaccinated HCW) (Table 3). However, systemic symptoms reported by controls were more often secondary to another health problem (most often respiratory or gastrointestinal symptoms), and only 15% of controls reporting such symptoms were validated. In contrast, for vaccinated HCW one third (76/281) of sys-temic events remained the primary diagnosis. While 27 vaccinated HCW (8%) who reported an eligible severe health event had a local reaction (Table 3), 21 of them missed work or consulted a physician for other health issues.Observed health eventsThe overall reporting rate for validated severe events was similar (p=0.7) between vaccinees (2.2%) and con-trols (2.3%) (Table 2). Among validated severe events 84.5% (361/427) prevented daily activities or resulted in work absenteeism alone and 15.4% (66/427) required a medical consultation with or without absenteeism. Most medical consultations were clinic visits (83.3%; 55/66), while 1.5% (1/66) were emergency department visits. At the time of follow-up, the reported problem had either resolved (64.4%; 275/427) or was improv-ing (28.5%; 122/427) in participants. 3.7% (16/427) of 7www.eurosurveillance.orgparticipants reported no change or worsening of their health problem, a proportion that was the same for both vaccinated HCW and controls, and 3.3% (14/427) did not answer this question. In the two seasons under study, one participant was hospitalised in the week fol-lowing vaccination for gastrointestinal symptoms that started six hours after vaccination. This individual was diagnosed with appendicitis resulting in an emergency appendectomy. No deaths were reported.Representativeness of events reportedStudy participants who did not respond to the 2012 online questionnaire (i.e. initial non-responders) were contacted by telephone. No difference in age (p=0.3) and sex (p=0.84) could be observed for con-trols between initial non-responders and responders. Vaccinee initial non-responders were slightly older than vaccinee online responders (p<0.001) but similar in sex (p=0.21) (Table 1). Overall, initial non-responders dif-fered by hospital occupation group (p<0.001). Vaccinated participants who responded online reported more health events (9.0%) across all types than vacci-nated non-responders (6.1%), although this difference was not observed among controls (Table 2). The rate of severe events, however, was generally similar in both responders and initial non-responders. Vaccinee non-responders reported not answering the online ques-tionnaire due to circumstantial factors, mainly because they reported being too busy (38.2%; 380/994), did not recall receiving the email (30.6%; 304/994), reported that the embedded link to the questionnaire did not work (11.9%; 118/994) or were away or did not check email regularly (8.1%; 80/994).DiscussionOnline monitoring offers an economical and sustainable platform to conduct large-scale electronic surveillance of vaccinated individuals, allows rapid identification of AEFI and minimises human resource needs. However, rapid large-scale surveillance of vaccine safety poses challenges which require a careful balance between information needs and feasibility. The quantity and validity of the information collected must be sufficient to allow stakeholders to detect and interpret safety signals in a timely manner, while requesting a minimal amount of information to obtain sufficient response rates from participants. Self-reported severe events offer the advantage of improved efficiency, but unless validated, may under- or overestimate AEFI reporting ratesTable 3Accuracy and validity of severe health events reported online by vaccinated and control healthcare workers, Canada, 2011–2012 (n=537 participants)Type of severe health eventControls Vaccinated healthcare workersSevere events reported or persons reportingNSevere events or persons followed-up by nurseNReporting errorsa or persons concernedNEligible eventsb,cor eligible persons N (%)Validated eventsc,d orperson concernedN (%)Severe events reported or persons reportingNSevere events or persons followed-up by nurseNReporting errorsaor persons concernedNEligible eventsb,cor eligible personsN (%)Validated eventsc,dor persons concernedN (%)Local reaction NA NA NA NA NA 31 31 4 27 (8) 6 (2)Systemic symptoms 76 74 19 55 (59) 11 (15) 267 255 20 235 (71) 76 (27)Allergy-like events 4 4 2 2 (2) 0 (0) 6 6 1 5 (2) 1 (<1)Respiratory symptoms 65 62 13 49 (52) 39 (55)e 158 153 9 144 (43) 114 (41)eGI symptoms 32 32 8 24 (26) 12 (17) 110 107 10 97 (29) 53 (19)ORS 34 27 9 18 (20) 0 (0) 42 42 3 39 (12) 6 (2)Paraesthesia 14 11 11 0 (0) 0 (0) 16 15 3 12 (4) 0 (0)Other 24 22 7 15 (16) 9 (13) 54 54 6 48 (14) 25 (9)Total personsf 149 134 40 94 (100) 71 (100) 388 366 33 333 (100) 281 (100)GI: gastrointestinal; NA: not applicable; ORS: oculorespiratory syndrome.a  The event reported was not considered, because it either did not prevent daily activities/work or require a medical consultation, or a medical consultation was pre-existing, or symptoms started prior to vaccination for vaccinees or prior to the reporting period for controls.b  Eligible events are events remaining after taking into account reporting errors.c  The denominators for the percentages are the total persons for the column in question.d  Validated events are events self-reported in online questionnaire that remained the primary diagnosis after a follow-up with a nurse. e  Significant difference between validated events in controls and vaccinees at p < 0.05.f Each person could report more than one health event, so the total number of persons is not equal to the total of reported events.8 www.eurosurveillance.orgOur results demonstrate that online safety surveillance can be used to effectively monitor influenza vaccine safety in a large number of vaccinees, despite the meth-odological limitations of relying on self-reported health events. As shown during the influenza A(H1N1)pdm09 pandemic, internet-based safety questionnaires are uniquely suited to rapid collection and analyses and can be adapted to provide monitoring for seasonal influenza vaccines [3,10-12]. The rapid collections of data, early in the mass vaccination campaigns that occur simultaneously across Canada allow for ongoing monitoring and analysis throughout the first weeks of activities and provides an opportunity to detect signals before widespread vaccine use. The ability of online surveillance to detect rare events will depend on the total number of respondents. Our study was able to detect events with a frequency of 1 per 1,000.Although public health officials were concerned about the possibility of an increase in oculorespiratory syn-drome among Agriflu recipients in 2012, event rates observed among HCW vaccinated in centres using this vaccine were similar to the rates observed in centres using other seasonal vaccines and to rates observed in the control group. This was later confirmed by passive surveillance results from the United States Vaccine Adverse Event Reporting System and the Canadian Adverse Event Following Immunization Surveillance System [4,13]. The 2012 interim analysis of data, in response to the temporary suspension of the Agriflu vaccine, confirmed that the network can provide timely evaluation of safety signals and adequately support decision makers. At this time, our network remains the largest able to provide active monitoring of influ-enza vaccine safety both nationally and internation-ally. Our findings confirm that influenza vaccines used in Canada for both the 2011 and 2012 seasons were safe and that their safety profiles were consistent with those expected following influenza vaccination.We also showed that most of the eligible severe events self-reported by vaccinated HCW were consistent with the nurse interviews and had indeed prevented daily activities, resulted in missed work or required a medi-cal consultation. The higher error rate in the control questionnaire reflects the difficulty controls may have in identifying the time period under surveillance and indicates a reference point, through a reminder email, may be needed for this group. Vaccinees have the advantage of a well-defined observation period start-ing at the vaccination event from which to start tracking any new or exacerbated symptoms. This discrepancy was particularly evident for the paraesthesia ques-tions where control symptoms starting more than one week before the questionnaire period were frequently reported, indicating background rates for chronic con-ditions or illness may be more difficult to separate from new events using an online questionnaire. This short-coming was addressed in the severe event follow-up where the difference between controls and vaccinees disappeared when more accurate questioning elicited precise event windows. Reassuringly, most primary diagnoses had indeed been reported by participants, but the main difficulty we encountered in validating health events reported by both controls and vaccinated HCW was in distinguishing the primary complaint from all other health events that occurred during the obser-vation period. This problem was particularly evident for local reactions and systemic symptoms, which often accompanied respiratory and gastrointestinal symp-toms, but which alone did not prevent daily activities or lead to absenteeism or medical consultations. The more specific events or symptom questions on the online questionnaire (respiratory symptoms, gastro-enteritis, etc.) were more likely to accurately capture a true event than nonspecific event or symptom ques-tions (fever, myalgia, etc.).The inclusion of a control group in our study is an added strength of the network. It provides background rates for health events just before the start of the influenza vaccination campaign in a similar popula-tion and enables precise calculation of risk estimates. Moreover, age and sex specific background rates can be estimated. Importantly background event rates can be compared over multiple years to address fluc-tuations in events or temporal variations, a potential weakness of the staggered data collection periods of controls and vaccinees.The similarity in severe event rates between initial non-responders and online responders indicates our online survey participants were representative of their respective vaccine and control groups. This suggests the rates of severe events elicited with our online sur-vey is representative of the group overall.LimitationsWe did not track the total number of individuals who presented at each institution for vaccination or the characteristics of those who were vaccinated but did not enrol in our study. Therefore, we cannot determine whether selection bias occurred at recruitment. Even if our sample is not representative of all HCW, we would not expect the rate of severe events to occur differen-tially among those who participated and those who did not. In our control group, we had fewer controls that were under the age of 30 years, but the proportion in the remaining age categories was similar, therefore we would not expect this to affect our estimates for severe events. Moreover our severe event rates mirror those seen in other studies collected by different methods [11,14].The importance of individual-level vaccine informa-tion became immediately apparent with the temporary suspension of one vaccine product. Fortunately for our study, only one among the healthcare centres consid-ered used multiple influenza vaccines, so we were able to infer which product individuals received based upon where they were immunised. However, our experience from institutional vaccination of HCW using a single 9www.eurosurveillance.orgproduct or vaccine lot may not hold true for children and adults vaccinated in the community. In subsequent years, as a wider range of vaccine products become available, individual-level vaccine product data will be necessary.HCW who participated in our surveillance constitute a unique group of vaccine recipients, which may not be representative of community vaccinees. Almost 70% of our participants have medical training or are involved in patient care. This likely enables them to better evaluate health problems and communicate chief com-plaints which may have improved the validity of the online survey. The validity of self-reported events by non-HCW populations may not be similar. Evaluation of this methodology in cohorts of children and non-HCW adults are needed.ConclusionsOnline surveillance can provide rapid assessment of influenza vaccine safety and is highly acceptable to the HCW participating in this activity. The addition of a control group enhances internal validity and estab-lishes background rates for common events of interest. This methodology works particularly well in a mass vaccination setting where large numbers of individuals can be rapidly enrolled and followed-up and meets the new enhanced surveillance requirements as outlined by the EMA.AcknowledgementsFunding source: The Public Health Agency of Canada and the Canadian Institutes of Health Research provided the fund-ing for this study. The funders had no role in this study. The authors gratefully acknowledge the expert assistance pro-vided by the Vaccine Evaluation Center, public health and hospital collaborators, the study site coordinators, research nurses and research staff. JAB is supported by a Career Investigator Award from the Michael Smith Foundation for Health Research.Conflict of interestJAB none. IR none. MCG none. WRB none. LV received re-search grants from Pfizer and Cubist, consultation fees from Cubist and is involved in sponsored clinical trials with Merck and Sanofi. OGV received research grants from Pfizer, Merck, and Sanofi and has participated in advisory boards for Novartis and Pfizer. JDK none. BLC none. SAM has received research grants from GlaxoSmithKline, Sanofi Pasteur and Pfizer. AM none. GDS received research grants from GlaxoSmithKline (GSK) and travel fee reimbursement to attend an ad hoc GSK Advisory Board without honorarium.Authors’ contributionJAB was involved in the conception and design of the study, acquisition of the data, the analysis and interpretation of the data and wrote the article. IR was involved in the conception and design of the study, the analysis and interpretation of the data, and writing and revision of the manuscript. MCG was involved in the acquisition of the data, the analysis and interpretation of the data, and writing and revision of the manuscript. WRB was involved in the design of the study, acquisition of the data and revision of the manuscript. LV was involved in the design of the study, acquisition of the data and revision of the manuscript. OGV was involved in the design of the study, acquisition of the data and revision of the manuscript. JDK was involved in the design of the study, acquisition of the data and revision of the manuscript. BLC was involved in the design of the study, acquisition of the data and revision of the manuscript. SAM was involved in the design of the study, acquisition of the data and revision of the manuscript. AM was involved in the design of the study, acquisition of the data and revision of the manuscript. GDS was involved in the conception and design of the study, ac-quisition of the data, the analysis and interpretation of the data, and revision of the manuscript.References1. European Medicines Agency (EMA). Explanatory note on the withdrawal of the note for guidance on harmonisation of requirements for influenza Vaccines and of the core SmPC/PL for inactivated seasonal influenza vaccines. London: EMA; 2014. Contract No.: EMA/CHMP/VWP/40560/2014. Available from: http://www.ema.europa.eu/ema/index.jsp?curl=pages/includes/document/document_detail.jsp?webContentId=WC500161022&mid=WC0b01ac058009a3dc2. European Medicines Agency (EMA). Interim guidance on enhanced safety surveillance for seasonal influenza vaccines in the EU. London: EMA; 2014. Contract No.: EMA/PRAC/222346/2014. Available from: http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/2014/04/WC500165492.pdf3. De Serres G, Gariépy MC, Coleman B, Rouleau I, McNeil S, Benoît M, et al.; PHAC-CIHR influenza Research Network (PCIRN). Short and long-term safety of the 2009 AS03-adjuvanted pandemic vaccine. PLoS ONE. 2012;7(7):e38563. http://dx.doi.org/10.1371/journal.pone.0038563 PMID:228029294. National Advisory Committee on Immunization (NACI). Statement on Seasonal Influenza Vaccine for 2013-2014. Canada Communicable Disease Report. 2013;39(ACS-4):1-37.5. National Advisory Committee on Immunization (NACI). Statement on seasonal influenza vaccine for 2011-2012. Canada Communicable Disease Report. 2011;37(ACS-5):1-55.6. National Advisory Committee on Immunization (NACI). Statement on seasonal influenza vaccine for 2012-2013. Canada Communicable Disease Report. 2012;38(ACS-2):1-36.7. National Advisory Committee on Immunization (NACI). Supplementary statement for the 2002-2003 influenza season: update on oculo-respiratory syndrome in association with influenza vaccination. Canada Communicable Disease Report. 2002;28(ACS-6):1-8.8. Novartis suspends distribution of seasonal flu vaccines Agriflu and Fluad in Canada as a precaution. Ottawa: Health Canada; 2012. [Accessed 9 Jun 2014]; Available from: http://www.healthycanadians.gc.ca/recall-alert-rappel-avis/hc-sc/2012/15096a-eng.php</eref>9. Health Canada pulls distribution of Novartis flu vaccines. Toronto: The Canadian Press; 2012 [Accessed 9 Jun 2014]; Available from: http://www.cbc.ca/m/touch/health/story/1.115681510. Lapphra K, Dobson S, Bettinger JA. Acceptability of Internet adverse event self-reporting for pandemic and seasonal influenza immunization among health care workers. Vaccine. 2010;28(38):6199-202. http://dx.doi.org/10.1016/j.vaccine.2010.07.019 PMID:2065466811. Härmark L, van Hunsel F, Hak E, van Grootheest K. Monitoring the safety of influenza A (H1N1) vaccine using web-based intensive monitoring. Vaccine. 2011;29(10):1941-7. http://dx.doi.org/10.1016/j.vaccine.2010.12.123 PMID:2123623512. Newes-Adeyi G, Greece J, Bozeman S, Walker DK, Lewis F, Gidudu J. Active surveillance for influenza vaccine adverse events: the integrated vaccine surveillance system. Vaccine. 2012;30(6):1050-5. http://dx.doi.org/10.1016/j.vaccine.2011.12.041 PMID:2220050113. Advisory committee on immunization practices (ACIP) summary report, June 19-20 2013. Atlanta: Centers for Disease Control and Prevention; 2013. [Accessed 16 Jan 2014]. Available from: http://www.cdc.gov/vaccines/acip/meetings/downloads/min-archive/min-jun13.pdf14. Iskander J, Haber P, Herrera G. Monitoring vaccine safety during an influenza pandemic. Yale J Biol Med. 2005;78(5):265-75. PMID:17132333


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items