Pharmaceutical Sciences, Faculty of
DSpace
Cooke, Kendrick
Pharmacovigilance is the reporting and documentation of adverse drug events. The effective tracking of these events is crucial for ensuring drugs in Canada are safe and effective for Canadians. Pharmaris Canada’s adverse event tracking from the Canada Vigilance database is done through manual data entry. This process is tedious, laborious, and prone to human error. The purpose of this project is to eliminate the need for manual data entry in pharmacovigilance for Pharmaris, an industry partner of UBCs, and to investigate how the implementation of an automated method would increase the efficiency and accuracy of the Canada Vigilance case assessment. An automated method was developed using R. The automated method proposed showed good accuracy compared to the manual case assessment process. A comparison between the differences of the methods was reported and found that there were some discrepancies between the two methods. The assessment of these methods was not accurate due to the differences in export data from the automated and manual methods. After adjusting the results for these time discrepancies, the automated method produced the same adverse reaction case assessment summary statistics as the manual method. By automating the assessment of the Canada Vigilance adverse event cases Pharmaris will be able to assess the cases much more efficiently, saving time and more accurately tracking Canada Vigilance adverse events.
1 DEVELOPMENT OF AN AUTOMATED PROCESS FOR PERFORMING PHARMACOVIGILANCE CASE ASSESSMENTS By Kendrick Cooke THE UNIVERSITY OF BRITISH COLUMBIA FACULTY OF PHARMACEUTICAL SCIENCES (Honours) April 2024 2 I. Abstract Pharmacovigilance is the reporting and documentation of adverse drug events. The effective tracking of these events is crucial for ensuring drugs in Canada are safe and effective for Canadians. Pharmaris Canada’s adverse event tracking from the Canada Vigilance database is done through manual data entry. This process is tedious, laborious, and prone to human error. The purpose of this project is to eliminate the need for manual data entry in pharmacovigilance for Pharmaris, an industry partner of UBCs, and to investigate how the implementation of an automated method would increase the efficiency and accuracy of the Canada Vigilance case assessment. An automated method was developed using R. The automated method proposed showed good accuracy compared to the manual case assessment process. A comparison between the differences of the methods was reported and found that there were some discrepancies between the two methods. The assessment of these methods was not accurate due to the differences in export data from the automated and manual methods. After adjusting the results for these time discrepancies, the automated method produced the same adverse reaction case assessment summary statistics as the manual method. By automating the assessment of the Canada Vigilance adverse event cases Pharmaris will be able to assess the cases much more efficiently, saving time and more accurately tracking Canada Vigilance adverse events. II. Lay Abstract Drugs can be associated with very promising capabilities to improve health and well-being. But drugs can also cause serious unwanted side effects that need to be monitored. If several individuals experience the same unwanted side effects of drugs, they no longer be accessible, to ensure patient safety. This is why side effects need to be monitored by drug companies. Effective systems need to be put in place to monitor these side effects. In this project, a method was proposed to help side-effect tracking by introducing an automated system to a generic drug company, Pharmaris Canada. This project aimed to improve the efficiency and accuracy of side-effect monitoring of cases for Canadians experiencing unwanted drug side-effects. The new system would save time, improve efficiency, and help Pharmaris Canada lessen the burden of tracking side effects. 3 III. Acknowledgments Firstly, I would like to thank Sonia Khalid and the team at Pharmaris Canada for allowing me to do my practicum at Pharmaris Canada. Through Sonia’s mentorship, I was able to learn a lot about pharmacovigilance and how the process works in Canada. I would also like to thank Dr. Anil Maharaj who was an incredible mentor to me. He has taught me so much about coding and consistently met with me throughout the term to troubleshoot my code. He was incredibly patient and helpful as I learned how to code in R and always made time for me to help me develop my project. Thank you to all my friends and family for providing me with continuous support throughout my entire project and supporting me through this process. 4 Table of Contents I. Abstract .................................................................................................................. 2 II. Lay Abstract ........................................................................................................... 2 III. Acknowledgments ................................................................................................. 3 Table of Contents .......................................................................................................... 4 IV. List of Abbreviations ............................................................................................. 5 V. List of Figures ........................................................................................................ 6 VI. List of Tables .......................................................................................................... 7 VII. List of Equations .................................................................................................... 7 1. Introduction ............................................................................................................ 8 1.1 Pharmacovigilance: Purpose and Historical Implementation ....................................... 8 The Purpose of Pharmacovigilance..................................................................................... 8 The History that Led to the Implementation of Adverse Drug Reaction Monitoring .............. 8 Overview of Pharmacovigilance .......................................................................................... 9 1.2 Pharmacovigilance as it Relates to the Pharmaceutical Industry .....................................10 1.3 Good Pharmacovigilance Practices – Specific Regulatory Requirements ........................10 Overview of Good Pharmacovigilance Practices ................................................................10 Periodic Summary Update Reports and Market Authorization Holder Internal Database ....11 1.4 The Canada Vigilance Program and Reporting ................................................................11 Background of Canada Vigilance Program .........................................................................11 The Use of Canada Vigilance in Pharmacovigilance ..........................................................12 1.5 Knowledge Gap of Canada Vigilance Case Assessment .................................................15 Knowledge gap: .................................................................................................................15 Aims and Hypothesis: ........................................................................................................16 2. Materials and Methods ........................................................................................ 16 2. 1 Resources and Materials Used .......................................................................................16 2.2 Project Overview .............................................................................................................16 2.3 File Structure ...................................................................................................................18 2.5 Step 1: Create an Adverse Reaction Term Dictionary ......................................................25 2.6 Step 2: Data Cleaning and Report Output........................................................................26 2.7 Quality Control Testing ....................................................................................................29 2.8 Application with Hypothetical New Drug and a Hypothetical Re-Run ...............................30 3. Results .................................................................................................................. 30 5 3.1 Automated Case Assessment Method Accuracy, Consistency and Error Assessment ....30 3.2 Application of Developed Automated Process to a New Drug Product .............................34 4. Discussion ............................................................................................................ 36 References ................................................................................................................... 40 Appendices .................................................................................................................. 42 Appendix A ..........................................................................................................................42 Appendix B ..........................................................................................................................42 Step 1 function: Step 1| ......................................................................................................42 Step 2 function: Step 2 .......................................................................................................42 Appendix C ..........................................................................................................................42 IV. List of Abbreviations CADR IS: Canada Adverse Drug Reaction Information System CV: Canada vigilance SAE: serious adverse event ADR: adverse drug reaction AE: adverse event BE: bioequivalent RA: regulatory affairs GVP: Good Pharmacovigilance Practices MAH: market authorization holder NOC: notice of compliance DIN: drug identification number PSUR: periodic summary update report AEN: adverse event number RMSE: root mean squared error 6 V. List of Figures Figure 1: Adverse Reaction Term Information Table from CV PDF Export Figure 2: Project Flow and Overview of the Project Structure Figure 3: File Structure of Sildenafil CV Folder Before Running the Code Figure 4: File Structure of Sildenafil CV Folder After Running Step 1 Figure 5: File Structure of Sildenafil CV Folder After Running Step 2 and Outputting the Final Report Figure 6: How to Search the Canada Vigilance Database Website Outline Figure 7: Export File by Selecting Standard View Figure 8: Webpage in the CV Export Process Showing the Number of Reports for the Search Criteria Figure 9: Final Selection Page Before Result Export: Export to an Excel Format Figure 10: CV Data Export File in Excel Format Figure 11: Representation of the Dictionary Created as an Output of the Step 1 Function Figure 12: Process of Dividing the CV Cases from the Export to Determine which Cases will Undergo Assessment Figure 13: Overall Expectedness Assessment of Cases Figure 14: Solifenacin PSUR Table Outputs with Manual Compared to Automated CV Assessment Method Figure 15: Sildenafil PSUR Table Outputs with Manual Compared to Automated CV Assessment Method Figure 16: Tadalafil PSUR Table Outputs with Manual Compared to Automated CV Assessment Method 7 VI. List of Tables Table 1: Key Points Included in Each Case from the CV for AE Assessment Table 2: PSUR Summary Table Example for Rosuvastatin Table 3: Example of Drugs Categorized as Specified or Unspecified Molecule Names Table 4: Three sections that need to be further manipulated to output all necessary case information needed for proper case documentation. Table 5: Lack of efficacy case assessment. Highlighted in pink is the new column added when the R script searches through AE terms and returns Y if any lack of efficacy term is found for that case. Table 6: Root Mean Squared Error (RMSE) Calculations across all summary PSUR metrics for solifenacin, tadalafil, and sildenafil. Table 7: PSUR Summary Table for Olmesartan New Drug Run Table 8: PSUR Summary Statistics for the Subsequent Year Run of Olmesartan VII. List of Equations Equation 1: RMSE calculation completed across all PSUR summary metrics and summation taken across all three drugs. 8 1. Introduction 1.1 Pharmacovigilance: Purpose and Historical Implementation The Purpose of Pharmacovigilance Medicines and vaccines have transformed the prevention and treatment of diseases. In addition to their benefits, medicinal products may also have side effects, some of which may be undesirable and/or unexpected (1). All medicines and vaccines undergo rigorous testing for safety and efficacy through clinical trials before they are authorized for use (2). However, the clinical trial process involves studying these products in a small number of selected individuals for a short period (3). Certain side effects may only emerge once these products have been used by a heterogenous population, including people with other concurrent diseases, and over a long period (2). Pharmacovigilance is a science relating to the detection, assessment, and prevention of adverse effects of any medicine/vaccine-related product (3). The main purpose of pharmacovigilance is to ensure that the risk-benefit ratio of drugs remains unchanged and that benefits outweigh the risks throughout the lifecycle of the product (3). Pharmacovigilance can additionally be used to study and investigate cases where a drug causes alternative unexpected benefits for a patient to expand its treatment potential (3). The History that Led to the Implementation of Adverse Drug Reaction Monitoring Pharmacovigilance was initially implemented worldwide after the Thalidomide crisis in 1961 (1). Thalidomide was approved for use as a sedative but was found to be very useful for pregnant women to relieve morning sickness symptoms (2). This new use for thalidomide with no further testing or tracking resulted in thousands of newborn babies being born with congenital abnormalities, often leading to death at an early age, from mothers who took thalidomide throughout their pregnancy (2). This crisis changed the world of pharmacovigilance. As a result, the implementation of spontaneous reporting of adverse drug reactions became systematic, organized, and regulated (1). In 1968, the World Health Organization program for International Drug Monitoring was implemented in Canada, Australia, the USA, the UK, Germany, Ireland, Sweden, Denmark, New Zealand, and the Netherlands. This program aimed to promote post-marketing surveillance and pharmacovigilance to improve all aspects of the safe use of drug products internationally (2). In Canada, in the 1990s the Canada Adverse Drug Reaction Information System (CADRIS) was launched for AE tracking(1). It became the Canada Vigilance 9 (CV) program in 2007 and aims to monitor any unwanted drug reactions experienced by Canadians (2). Overview of Pharmacovigilance Health products include drugs, medical devices, and natural health products (3). More specifically, drugs include both prescription and nonprescription pharmaceuticals; biologically derived products such as vaccines, serums, and blood-derived products; cells, tissues, and organs; disinfectants; and radiopharmaceuticals (4). Both generic and brand-name drugs in Canada must follow pharmacovigilance guidelines of reporting serious adverse events (SAE), adverse drug reactions (ADR), adverse events (AE), unusual failures in efficacy, and preparing yearly reports (4). An ADR, in contrast to an AE, is characterized by the fact that a causal relationship between the drug and the occurrence of an unfavorable symptom is suspected (4). Whereas an Adverse Event (AE) is any unfavorable and unintended symptom (4). Therefore, all ADRs are considered AEs but would be further categorized as ADRs if there is a potential causal relationship between the unwanted symptom and the drug (3). Additionally, an SAE is an event that results in hospitalization, life-threatening health concerns, congenital abnormality, disability, or death (3). AEs may occur under normal use conditions of the product. Reactions may be evident within minutes or years after exposure to the product and may range from minor reactions like a skin rash to serious and life-threatening events such as a heart attack or liver damage (3). Generic drugs contain the same medicinal ingredients as the brand-name drug meaning it is considered bioequivalent (BE) to the reference product (3). There may be many generic versions of the same reference product (4). A brand name product is the innovator product; therefore, it is the first marketed product of that molecule in a specific country. Brand name products acquire a patent and/or exclusivity once they receive their Notice of Compliance to be able to market and distribute the drug in Canada (4). After the patent expires, generic companies can manufacture and sell BE generic versions of a brand name product upon receiving approval from Health Canada (3). 10 1.2 Pharmacovigilance as it Relates to the Pharmaceutical Industry Pharmacovigilance as a sector relates to several branches of the pharmaceutical industry including clinical sciences, regulatory affairs, manufacturing, quality assurance/quality control, and marketing and distribution (4). Firstly, the clinical sciences branch of industry is responsible for BE and clinical studies to establish the safety and efficacy of drug products. This information forms the basis of the product dossier, which is submitted for approval by the Regulatory Affairs (RA) department. The regulatory affairs (RA) branch of the pharmaceutical industry obtains and maintains approval for a drug product that is sold in Canada (7). This branch works directly with the pharmacovigilance department to submit updated product monographs and other safety-related reports if sought by regulatory agencies. Also, they partner with the pharmacovigilance department in the triaging of the complaints received and take necessary regulatory action (3). Further, in addition to the regulatory affairs department, quality assurance and quality control sectors work alongside the pharmacovigilance department to address any adverse events associated with a product quality defect (4). Lastly, Sales & marketing, and distribution of health products are relevant to the pharmacovigilance department because they are in touch with consumers, doctors, pharmacists, and patients and relay complaints to the regulatory and pharmacovigilance departments (4). 1.3 Good Pharmacovigilance Practices – Specific Regulatory Requirements Overview of Good Pharmacovigilance Practices To comply with pharmacovigilance practices there exists a strict set of Health Canada guidelines, The Good Pharmacovigilance Practices (GVP), that gives guidance to drug companies on their expectation for reporting and AE tracking (4). The GVP is a set of guidelines and regulatory requirements for manufacturers and importers to report adverse drug reactions and unusual failures in efficacy to Health Canada (4). The companies that are responsible for upholding these regulations and adhering to Health Canada guidelines are the Market Authorization Holders (MAH) and importers (4). An MAH is a company or entity that has the legal right to a product and holds a Notice of Compliance (NOC) or the Drug Identification Number (DIN) which gives the right to distribute a health product throughout Canada (4). These MAHs can either hold the right to distribute their brand name product or a generic product in Canada (4). The MAH is responsible for the evaluation of AEs, the reporting of AEs to Health Canada, 11 performing literature searches, preparation of Annual Summary Reports, the preparation of issue-related summary reports, and reporting unusual failures in the efficacy of new drugs (4). Periodic Summary Update Reports and Market Authorization Holder Internal Database Pertinent to this project, an MAH must upkeep an internal database and create Periodic Summary Update Reports (PSURs). The internal database must contain all reported AE cases pertinent to any drug molecule that an MAH can market and distribute in Canada (4). A PSUR is a summary document that provides summary statistics of cases included in the internal database. PSURs must be completed by the MAH each year for each drug that they are authorized to sell in Canada (4). Both require a complete assessment of all CV cases in the database for that molecule over the reporting period (4). 1.4 The Canada Vigilance Program and Reporting The CV Program is used in pharmacovigilance as a resource for the creation of both the internal database and PSURs. The CV Program is a program that outlines post-marketing surveillance and enables Health Canada to monitor safety profiles once drugs are marketed and distributed throughout Canada (4). The program allows for a centralized resource for drug companies to access and assess AEs and for Canadians to report cases of unexpected drug symptoms (5). AEs are submitted by consumers and health professionals who submit reports voluntarily as well as manufacturers and distributors who are required to submit reports (5). Background of Canada Vigilance Program The Canada Vigilance Program encompasses a CV database, that has collected adverse event reports since 1965 (5). The database is medDRA coded for simplicity of analysis. MedDRA coding is a system that streamlines the reporting of adverse events by dividing all adverse events into system organ classes that are then subdivided into grouped terms by level (6). The system works to provide one proper term that corresponds to an adverse event. For example, if the report source reported a patient experienced a stomach ache and abdominal pain, MedDRA coding will streamline these symptoms so that one single term is used to improve data analysis and promote cohesiveness (6). 12 The CV database can be searched by drug product or active ingredient with a specific date range and will pull up any reported AEs that fall under the conditions set by the user for analysis. When searching by active ingredient the CV database will export any cases that list the active ingredient as a suspect molecule (5). CV uses the terms suspect and concomitant to describe molecules in a case. A suspect molecule is any drug that could have a causal relationship with the adverse event experienced. A concomitant drug is a drug that the patient was taking alongside the suspect molecule at the time that the AE was experienced (5). The database stores relevant AE information for all cases including case source, concomitant medications a patient was taking at the time of the event, descriptions of MedDRA-coded adverse events experienced, seriousness, and patient information such as age, weight, height, and gender. This information can be exported for analysis directly from the database to an Excel or PDF format (5). The Use of Canada Vigilance in Pharmacovigilance It is Health Canada regulation that the MAH within a reporting period includes all AE reports that exist in the Canada Vigilance (4). These AE reports contribute to their yearly PSUR and their internal database (4). Denoted in Table 1 are the key pieces of information needed by the MAH to evaluate an AE report. Table 1: Key Points Included in Each Case from the CV for AE Assessment Key Points to Include in the Case Case Case Identification # 123456789 Patient Identifier(s) F, 85yrs, 42kg, 167cm Suspect Product ROSUVASTATIN Adverse Reaction Term Nausea Report Source MAH Concomitant Medications Ibuprofen, Metformin Indication(s) Hyperlipidemia Seriousness Y Label use or Off label use? Label Lack of Efficacy? N Expectedness Y Key points included in a case from the Canada Vigilance Database to be recorded by the MAH. This is not a comprehensive list of all aspects, just key aspects. Table made with information from (5). The key points highlighted in blue, denote information that is taken directly from the database. Whereas the remaining columns in white need to undergo further analysis to be completed. Case 13 identification numbers in the CV database are typically referred to as Adverse Event Numbers (AEN) and are 9 digits long (5). Patient identifiers are listed including sex, age, weight, and height. The Suspect Product section outlines the product that is presumed responsible for the experienced adverse reaction, in this case, Rosuvastatin. Next is the MedDRA-coded AE term, and lastly the report source which could include MAH, Community, Hospital, etc. Then, the concomitant medications and indications are listed, and the case is specified whether it is attributed to an SAE. The first category to undergo further analysis is the label use determination which pertains to the indications. Indication is the reason the consumer is taking the drug (e.g., for what medical condition or symptom). In this case, the patient is taking Rosuvastatin for hyperlipidemia. Since this is an approved use for Rosuvastatin, it is categorized as a label use. Next, is a lack of efficacy assessment which determines if the AE symptoms experienced were related to a lack of efficacy of the suspect molecule. Lack of efficacy terms will be listed in the AE terms section of the case. Finally, an expectedness assessment is done. Expectedness is a term referring to the adverse event experienced by the individual – an expected adverse event is listed in the drug’s product monograph. A product monograph is a document that outlines information on manufacturing, dosing, as well as health-related and symptom-related issues of a drug product marketed in Canada. If a term is listed as a symptom on the company-specific product monograph it is referred to as expected (3). Take the case above in Figure 1 for example – this is an image taken from a PDF export of the CV database for Sildenafil. Sildenafil is primarily used for the treatment of pulmonary hypertension and erectile dysfunction. Figure 1: Adverse Reaction Term Information Table from CV PDF Export Figure 1: Example of a case with multiple adverse reaction terms listed. Image taken from CV database export in PDF format for ease of visualization (7). The table labeled Adverse Reaction Term Information is present in every single case listed in the export. In this particular case, the patient experienced three adverse events: arthralgia, which is joint pain; blood pressure inadequately controlled; and blood pressure increased (8). If 14 one or more of the Adverse Reaction terms were assessed as unexpected meaning, they were not listed in the product monograph the overall assessment would yield unexpected. If all adverse events experienced by the individual were listed in the product monograph the case would be overall assessed as expected (3). For generic drug companies, a key consideration when analyzing CV cases is whether the case pertains to their molecule. Therefore, generic companies need to know whether an AE was experienced with the use of a specified molecule or an unspecified molecule (3). An unspecified molecule name case describes an AE where the suspect molecule is the generic name, such as what is listed above in Table 1. This case has ROSUVASTATIN as its suspect molecule. Therefore, generic drug companies would need to assess this case to contribute to their PSUR and their internal database. This is because this case could be attributed to their molecule as no brand or company name is associated with the product. In other cases, in place of ROSUVASTATIN, a brand name such as Crestor or ACT-ROSUVASTATIN may be listed (3). ACT denotes a manufacturer abbreviation associated with a generic drug company in Canada. The prefix specifies that the molecule associated with that case is attributed to a specific generic drug company with that manufacturer abbreviation (9). These cases would be categorized as specified molecule name cases and do not need to undergo an assessment by generic companies because the specified molecule is not their product. Pharmaris Canada, located in Surrey, BC, serves as a key industry partner with UBC. They market and distribute a variety of prescription generic health products across Canada, with further expansions into South America, under the Pharmaris name (9). As a generic drug company, Pharmaris Canada is required to conduct case assessments for CV case reports pertaining to their molecule name or an unspecified molecule case. Pharmaris Canada uses PRZ as their manufacturer abbreviation to identify their product name in Canada. For example, Pharmaris must assess any cases with ROSUVASTATIN and PRZ-ROSUVASTATIN as the suspect molecule. For Pharmaris, the AE case information (seen in Table 1) for all unspecified molecule AE reports alongside an expectedness, label use and lack of efficacy assessment must be included in the PSUR as well as a summary table. Table 2 denotes an example of a PSUR summary table from the data of the CV case assessment. This outlines all of the key aspects that need to be explicitly mentioned in this table for the completion of a PSUR by Pharmaris. 15 Table 2: PSUR Summary Table Example for Rosuvastatin ROSUVASTATIN Canada Vigilance Online Database 2022-09-19 to 2023-05-31 Number of Case Reports 75 Specified Case Reports 50 Unspecified Case Report 25 Duplicate Case Report 1 MAH as the source of the report 24 Serious Reports 15 With Concomitant Medication 14 Medically important conditions 12 An example of a PSUR summary table. 1.5 Knowledge Gap of Canada Vigilance Case Assessment Knowledge gap: Pharmaris Canada takes on the burden of completing the documentation and assessment for all Canada Vigilance database AE reports for every PSUR they create. These reports in a year can be anywhere from 20 to over 200 AE reports for each drug. Pharmaris Canada uses a manual data entry process to complete these cases. The current procedure is to import the present year's AE reports, for a particular drug’s PSUR by searching by molecule and exporting it into a PDF document. The case report details in the document are then manually entered into Pharmaris’ internal database including all key AE report aspects outlined in Table 1. There are available commercial Pharmacovigilance software like Argus, and ARISg that can efficiently do the database extraction and case assessment that larger companies will purchase to lessen the workload on their department (10). Since Pharmaris is a smaller pharmaceutical company, cost is a principal consideration detracting from the use of proprietary pharmacovigilance software. The process of manual CV case assessment is laborious and tedious work. Depending on the number of cases, it can take an employee anywhere from a few days to a week to complete all of the cases thoroughly and assess them. This manual process needs to be conducted each year for every drug that Pharmaris sells in Canada. The automation of this time-consuming manual data entry could significantly improve the efficiency of this process and improve the accuracy of CV case assessment. The focus of this project is to simplify and automate the process of assessing AE reports from the Canada Vigilance online database for input in PSURs and to contribute to Pharmaris’ internal database. 16 Aims and Hypothesis: This project aims to evaluate the accuracy and feasibility of implementing an automated pharmacovigilance CV case assessment system as a replacement for manual data entry. It is hypothesized that he developed method will improve the accuracy, consistency, and efficiency of annual pharmacovigilance reports and case assessments compared to the manual data entry method. 2. Materials and Methods 2. 1 Resources and Materials Used The automated process proposed uses the CV AE case information as the input, which is then processed using R and its associated integrated development environment (IDE). The Canada Vigilance database is a free online database run by Health Canada that was used to access the AE information from the following link: https://cvp-pcv.hc-sc.gc.ca/arq-rei/index-eng.jsp. R (R 4.3.2, R Statistical Software, Vienna, Austria) is a programming language and software for statistical computation often used in academia in data science, statistics, and research areas (11). The IDE is an extension of R that provides an easy-to-use interface for users to easily organize their code and manage projects using the openxlsx (version 4.2.5.2), writexl (1.5.0), readxl (version 1.4.3), tidyverse (version 2.0.0), purrr (version 1.0.2), dplyr (version 1.1.4), data.table (version 1.15.0), plyr (version 1.8.9) packages (11). 2.2 Project Overview This project automates CV case assessment through a two-step method in R. Figure 2 illustrates the overall project overview. To begin the user must export a CV case file for their desired reporting period and drug molecule. Specific details on how to export files from CV are elaborated on in section 2.4. 17 Figure 2: Project Flow and Overview of the Project Structure Project Overview. Start with a Canada Vigilance database search to obtain a data file export discussed in section 2.4 (green). Then proceed to create an adverse reaction term dictionary for case assessment, discussed in section 2.5 (yellow). The last step of the process was to use the export and the dictionary files created in step 1 to create a final report discussed in section 2.6 (blue). Step 1 creates an adverse reaction term dictionary. This dictionary is needed to assess the cases for expectedness. A user-defined expectedness analysis must be done after running step 1. A user-defined expectedness analysis is done by comparing the AE terms present in the dictionary output of step 1 to the product monograph. If the AE term is listed in the product monograph it is assessed as expected, otherwise is it considered unexpected. The final result of this step is a completed dictionary with all the AE terms and their associated user-defined expectedness analysis for that drug. The completed dictionary, the CV data export, and a list of drug indications listed in an Excel document are used as inputs for Step 2 which creates and outputs a final report in an Excel format. To conduct each step, separate computational functions were developed in R, which simplifies the end-user implementation of this project because functions allow for efficient portability (12). 18 2.3 File Structure The file structure for this project is essential for the simple and effective organization of the inputs and outputs to ensure that R can read, manipulate, and summarize CV data appropriately. The file structure for this project consists of a file folder designated to each drug that Pharmaris markets and sells in Canada. Within this folder there 4 subfolders and an indication Excel document to organize the different inputs and outputs necessary for the functions to run correctly. Figure 3 is an example of a folder for Sildenafil. Before the code is run, Figure 3 depicts which files each folder should contain and the organization before running the code after the CV database export has been completed and the file has been saved in the CV Data subfolder. Figure 3: File Structure of Sildenafil CV Folder Before Running the Code File Structure where Sildenafil CV is an example of a drug’s file folder. Where CV export, functions, Dictionary, and Reports are all subfolders within the folder and export.xlsx, Step 1, Step 2, and previous years' reports all exist in their corresponding subfolder. After the CV export has been saved in the CV Data folder, the Step 1 function is run. The step 1 function must be opened through the opening functions folder to ensure relative paths function correctly. After Step 1 has been run the file structure will resemble that of Figure 4 below. 19 Figure 4: File Structure of Sildenafil CV Folder After Running Step 1 File Structure where Sildenafil CV is an example of a drug’s file folder. Where CV export, functions, Dictionary, and Reports are all subfolders within the folder and export.xlsx, Step 1, Step 2, and previous years' reports all exist in their corresponding subfolder. After Step 1 is run there is a step for a user-defined expectedness analysis that must be completed before continuing to report output. This manual case assessment is discussed in detail in section 2.5. Lastly, the Step 2 function is run to create the final report and the files that exist in the file folder will resemble the structure above in Figure 5. 20 Figure 5: File Structure of Sildenafil CV Folder After Running Step 2 and Outputting the Final Report File folder for sildenafil with all existing files after running step 2 of the R scripts. Figure 5 describes which files will exist in corresponding subfolders after running step 2, this function must be opened by opening the functions folder to ensure relative paths operate correctly. The Excel report document will then be used to contribute to the Pharmaris’ internal database and will contribute to the PSUR for that reporting period. The way the file folder is organized it is already ready to be run again the following year and no additional manual file organization that required before the code can be run again. The dictionary document will remain in its subfolder and be built upon each year that the code is run so that case assessment for a specific adverse reaction term never has to be repeated. This file structure organization will allow the R functions for steps 1 and step 2 to use relative paths to read and write the files. The use of the file structure and functions with relative paths allow for the easy portability of this automated method. 2.4 Canada Vigilance Database Search An input for the automated process is the data pulled from the CV database, this section explains how to export the data from the database. CV data searches are done yearly, these searches are then exported to create an export.xlsx as described below. As seen in Figure 6, the first option is to input either the Initial Received Date or the Latest Received Date. 21 Canada Vigilance Database search criteria, the four section of search criteria that are filled in as listed above in order to search information in the database. This figure was taken from Health Canada, Canada Vigilance Online Database (Canada, 2008). Figure 6: How to Search the Canada Vigilance Database Website Outline 22 For PSUR purposes Initial Received Date is selected. The second input in Section 1 is the date, this simply is the reporting period, for instance in this case the reporting period for PSUR is always a year. For example, for sildenafil, the reporting period is from November 30th, 2022, to November 29th, 2023. Hence, the database will output all cases within that date range. In Section 2 no options need to be selected, it can be left as the default. In the third section, the search needs to be specified and searched by Active Ingredient, once selected the drug molecule that you are searching for will need to be typed into the keyword section. In this case, SILDENAFIL is being searched. After entering the drug molecule name, Find Health Products should be selected, and from the drop-down menu. Select All Below should be selected. Lastly, in Section 4, similar to Section 2, it can be left as the default. Figure 7 illustrates the following page after selecting search, standard view is the default, and the user should press continue. Figure 7: Export File by Selecting Standard View Canada Vigilance Database report display taken from Health Canada, Canada Vigilance Online Database. The standard view is to be selected and the user should press continue to progress to the data export (Canada, 2008). 23 Figure 8: Webpage in the CV Export Process Showing the Number of Reports for the Search Criteria Canada Vigilance Database Search Criteria to export results taken from Health Canada, Canada Vigilance Online Database. The user should select export results (5). Figure 9: Final Selection Page Before Result Export: Export to an Excel Format Canada Vigilance Database export results options where in this project, data should be exported in an Excel Format, by selecting Seach Results Export (Excel Version) and selecting Export Results. Image taken from Health Canada, Canada Vigilance Online Database (Canada, 2008). At this point, an Excel file will be downloaded for all adverse events reported from November 30th, 2022, to November 29th, 2023, for all cases with sildenafil as the suspected molecule that could have had a causal relationship with the adverse event experienced. The 24 exported Excel document is approximately 66 columns with approximately 2,500 rows that will differ based on the number of cases. Several duplicate rows exist in the data frame as well as several repeats of the same AE number. Although the adverse event number is repeating the subsequent rows have different AE terms associated with them or different concomitant medications. For example, this data export has 2,408 rows of data that depicts 107 unique adverse reaction numbers or cases. The way that the data is exported from CV is that if there are multiple list concomitant medications for one case then that AE number will be listed in the number of consecutive rows for how many concomitant medications are recorded. The same repetition will occur if a person is using the medication for multiple listed indications or if there are several reported AE terms. Figure 10 illustrates how this would be present in the data export to provide an example. Figure 10: CV Data Export File in Excel Format Example of AE number repetition with multiple AE terms or concomitant medications (9). The automated process should be able to address these duplicate rows. Once the file has been exported and saved in the file folder under the subfolder CV Data the Step 1 function can be run. 25 2.5 Step 1: Create an Adverse Reaction Term Dictionary The Step 1 function is responsible for building an adverse reaction term dictionary. The dictionary will remove duplicate adverse event terms and allow for user input to assess the expectedness of the case based on a comparison to the drug’s product monograph. This dictionary eliminates the need for manual expectedness assessment for every individual case one at a time but does still require some manual user input. A benefit of the dictionary method is that once one adverse event term is assessed it will not need to be assessed again in subsequent years. The function first loads in the CV data export to R. Then, all adverse reaction terms are isolated, and duplicate terms are removed. The output of the function is automatically opened as an Excel file demonstrated in Figure 11. An additional step built into this function is if there is no dictionary and this is the first run of this function for a given drug, a new dictionary document will be created adding all terms to it and highlighting terms that need to be assessed manually as seen in Figure 11A. Figure 11: Representation of the Dictionary Created as an Output of the Step 1 Function If there is a pre-existing dictionary, it will be updated. Only new terms will be added and highlighted at the top of the list as seen in Figure 11B. The Excel file opens with two columns, one labeled ‘AE_terms’ and the second labeled ‘Y.’ This is where user-defined expectedness analysis must be completed before proceeding to Step 2. At this stage, the employee completing the case report for the CV data export will now have to go through the list of highlighted adverse terms and (A) Adverse reaction term dictionary exported after Step 1 that has been run for a new drug run with no pre-existing dictionary, all adverse reaction terms are highlighted for assessment. (B) subsequent year run adverse reaction term dictionary with new terms highlighted at the top of the document for assessment. 26 referring to the product monograph will place a ‘Y’ in the Y column if the side effect is expected with the corresponding drug, based on a comparison with the product monograph. Once complete for all highlighted rows – the employee will save the document in the dictionary folder and then can continue to run Step 2. 2.6 Step 2: Data Cleaning and Report Output Step 2 was formulated to format the CV export and produce PSUR information and case information to be included in Pharmaris’ internal database in a format congruent with Pharmaris’ reporting standards. As mentioned in Section 2.4 the CV export has several repeat rows of the same adverse event numbers for indications, AE terms, and concomitant medications. First, rows from the CV export where the Product Role was specified as Suspect were isolated, and cases were separated based on specified or unspecified molecule names, as seen in Figure 12. Figure 12: Process of Dividing the CV Cases from the Export to Determine which Cases will Undergo Assessment Splitting of cases based on specified or unspecified molecule names to be assessed in future steps. This step was performed first because Pharmaris only has to include cases that could be attributed to their molecule in their CV AE analysis. So, the unspecified molecule names and cases with PRZ are isolated from the export to undergo further assessment. To illustrate the difference in specified and unspecified molecule name cases, in Table 3 sildenafil and sildenafil 27 citrate product descriptions were isolated from the CV export. All specified molecule names cannot be attributed to the Pharmaris’ molecule and do not need to undergo any assessment. Table 3: Example of Drugs Categorized as Specified or Unspecified Molecule Names Filtering Out Cases that Need to be Assessed Adverse Reaction (AER) Number Product Role Product Description E2B_06867789 Suspect SILDENAFIL E2B_06867790 Suspect SILDENAFIL CITRATE E2B_06867791 Suspect REVATIO E2B_06867792 Suspect ACT-SILDENAFIL The section in Table 3 shaded in pink is the set of cases that will be further assessed using the case assessment and data-cleaning steps listed below. The final report does not need to include an assessment of specified molecule name cases but must include a count of the total number of cases and the number of specified molecule name and unspecified molecule name cases for the PSUR summary table. The next step is to take all unspecified molecule name cases and perform the necessary case assessments to contribute to the final report. The duplicates due to concomitant medications listed were collapsed and listed into a single row separated by commas under a Concomitant Medications column. The function then needs to perform 3 assessments including, lack of efficacy, indication, and expected assessments (Table 4). Table 4: Three sections that need to be further manipulated to output all necessary case information needed for proper case documentation. Lack of Efficacy Indication Expectedness Cases need to be assessed for whether the case involves a lack of efficacy. Cases need to be assessed for whether the drug is being used for Label use, off-label use, or an unknown use. Cases need to be assessed whether the adverse event experienced is expected or unexpected. First, the lack of efficacy assessment was done (Table 4). Lack of efficacy terms appear in the adverse event column of the CV export. Lack of efficacy contains several medDRA-coded terms. These terms are drug ineffective, therapeutic product effect incomplete, therapeutic 28 product ineffective, loss of therapeutic response, therapeutic product effect decrease, and drug ineffective for approved indication. Now using the list of unspecified molecule name cases, the function will then search through these cases for any of the above lack of efficacy terms. If any term is found for a specific case it will be listed as a lack of efficacy case, as seen in Table 5 Table 5: Lack of efficacy case assessment. Highlighted in pink is the new column added when the R script searches through AE terms and returns Y if any lack of efficacy term is found for that case. Lack of Efficacy Case Assessment Adverse Reaction (AER) Number Adverse Reaction Terms Lack of Efficacy E2B_06867789 Therapeutic product ineffective Y E2B_06867790 Headache N E2B_06867791 Drug Ineffective Y E2B_06867792 Loss of therapeutic response Y Next, is an indication assessment. A case is assessed based on the reason that the patient is taking the medication of interest. A case can be assessed as either off-label use, label use, or unknown use. The indications.xlsx document is loaded into R and used as an input. This document consists of a list of approved indications for the drug that can be found in the product monograph. If the indication listed for a case is present in the indications.xlsx document, it will output as a label indication. If there is no indication listed for that case or it is listed as a product used for an unknown indication, then Unk will be outputted. Lastly, if neither case is true, the case will be labeled as off-label use. Lastly, the cases will be assessed for expectedness. This step of the process is where the user-modified data dictionary is used as input. The data dictionary is read into R, if the term is present in the data dictionary with a Y in the second column, indicating it is expected, it will output as expected, otherwise, it will be assessed as unexpected. If there are multiple AE terms listed per case, the overall case assessment will only output as expected if all AE terms are expected otherwise the overall case will be unexpected (Figure 13). 29 Figure 13: Overall Expectedness Assessment of Cases Adverse Reaction Terms are assessed based on expectedness and the function outputs the expectedness column based on the terms present in the adverse reaction terms column. The resulting output after running this section of code is an additional column that uses the procedure above to assess the cases by referring to the dictionary. At this stage when all case assessment aspects are completed there is some cleaning required for the removal of unnecessary columns. The case information assessed is all of the information presented in Table 1 and is used to contribute to the PSUR. Pharmaris needs to keep additional pieces of information taken directly from the CV export for tracking purposes in their internal database. This has been built into the step 2 function that adds all additional information required by Pharmaris. Lastly, a PSUR summary data table is made using the information from the case assessment steps to output the summary metrics for a PSUR. For simplicity of output, the merged database will automatically output an Excel document in the reports subfolder of the drug folder. There are four sheets included in the output, one including both specified molecule name and unspecified molecule name cases, the second with just unspecified molecule name cases to add to Pharmaris’ database, the third is the PSUR summary table, and the fourth with the raw data to be kept on record as per Health Canada guidelines. To see an example of this output refer to Appendix A. 2.7 Quality Control Testing To ensure that all pieces are running correctly, a quality control test has been performed. This test will run for Tadalafil, Sildenafil, and Solifenacin for the 2022-2023 reporting period. This output will then be compared to the manual data entry outputs that were completed for last year's PSURs. These findings will then be compared, and the root mean squared error (RMSE) will be reported with any discrepancies within the manual data entry approach and the automated approach for all metrics in the PSUR summary table across all three drugs. This will be calculated using the Equation 1 figure below. 30 2.8 Application with Hypothetical New Drug and a Hypothetical Re-Run To demonstrate the utility of this approach for a new drug the code will be run for Olmesartan. A report will be created using the automated method for Olmesartan for the 2022-01-01 to 2022-12-31 reporting period. The user will input the indications associated with Olemsartan in the indciaions.xlsx document in the drugs file folder. Then the step 1 function will be run, and the user-defined expectedness analysis will be done to complete the dictionary. Then step 2 will be run to output the report. Following this process, the code will be run in a subsequent year for Olmesartan over the reporting period of 2023-01-01 to 2023-12-31. The outputs displayed will be both of the PSUR tables for the initial run and the subsequent year run. Additionally, the number of AE terms that needed to undergo user-defined expected analysis will be noted for both the new drug run and the subsequent year run to evaluate the efficacy of the dictionary update with subsequent year runs. 3. Results To view the completed functions for both Step 1 and Step 2 please refer to Appendix B. 3.1 Automated Case Assessment Method Accuracy, Consistency and Error Assessment A comparison of the manual and automated case assessment methods across PSUR summary statistics was done for all three drugs. Then, an error assessment is reported for the RMSE values for all summary statistics across all three drugs to show the accuracy of the automated method compared to the manual method. For all PSUR summary statistics, a bar graph is presented to visualize the resulting data. The total cases in Figures 14, 15, and 16 are in subplot A which includes both specified molecule name case reports and unspecified molecule Equation 1: RMSE calculation completed across all PSUR summary metrics and summation taken across all three drugs. 31 name case reports. Since Pharmaris only needs to assess cases that could be attributed to their molecule, the further analysis of summary statistics in subplot B of Figures 14, 15, and 16 is done only with unspecified molecule name cases and PRZ molecule cases. Figure 14 shows a comparison between the new proposed automated method and the manual method for solifenacin. Both methods detected the same overall number of cases, as seen in Figure 14A. Figure 14: Solifenacin PSUR Table Outputs with Manual Compared to Automated CV Assessment Method The manual method bars are in blue and the automated method counts are in orange. PSUR summary data was calculated from CV data from the reporting period of 2022-11-13 to 2023-07-31. (A) Denotes PSUR summary statistics accounting for all overall cases for solifenacin. (B) Denotes PSUR summary statistics for only unspecified molecule name cases for solifenacin. However, the automated method did detect one less specified case compared to the manual method. In Figure 14B this difference of one case is carried across all other PSUR summary metrics. Resulting in a discrepancy of one case for all metrics in 14B except both methods found no duplicate cases. Although some discrepancies were observed, the statistics only differed by one case. Figure 15 shows a comparison between the new proposed automated method and the manual method for sildenafil. The manual method detected one additional case; this case was for an unspecified molecule. The number of specified cases was the same (Figure 15A). 32 Figure 15: Sildenafil PSUR Table Outputs with Manual Compared to Automated CV Assessment Method The manual method bars are in blue and the automated method counts are in orange. PSUR summary data calculated from CV data from the reporting period of 2022-09-21 to 2023-05-31. (A) Denotes PSUR summary statistics accounting for all overall cases for sildenafil. (B) Denotes PSUR summary statistics for only unspecified molecule name cases for sildenafil. In Figure 15B, the assessment of unspecified cases was similar. However, there were differences. A discrepancy of one case between the manual and automated method for the number of duplicate cases was due to the presence of an additional unspecified case identified by the manual entry. A discrepancy of three cases between manual and automated processes for the number of cases with concomitant medications was observed. Although some discrepancies were observed, the majority of characteristics in Figure 15B showed identical results for both manual and automated assessment methods. Figure 16 shows a comparison between the manual and automated methods for tadalafil. Both the manual and automated methods detected the same number of case reports as seen in Figure 16A. 33 Figure 16: Tadalafil PSUR Table Outputs with Manual Compared to Automated CV Assessment Method The manual method bars are in blue and the automated method counts are in orange. PSUR summary data calculated from CV data from the reporting period of 2022-09-19 to 2023-05-31. (A) Denotes PSUR summary statistics accounting for all overall cases for tadalafil. (B) Denotes PSUR summary statistics for only unspecified molecule name cases for tadalafil. Although the total number of cases reported was the same, the breakdown of the number of specified and unspecified differed by one case. The manual method concluded one less specified case and one more unspecified case than the automated method. In Figure 16B a discrepancy of cases was observed for duplicate cases, with one case detected in the manual method and none in the automated method. There were no other discrepancies in any of the other observed statistics in Figure 16B. The automated method showed the same results as the manual method. The root mean squared error (RMSE) was calculated across all 3 drugs for all PSUR summary metrics separately and reported below. The RMSE measures the average differences between the observed values in the manual method and the values outputted from the automated methods. It will illustrate, on average how many cases the results from the two methods differ by across all three drugs. The RMSE values for all PSUR summary metrics were between 0.5 and 2 (Table 6). 34 Table 6: Root Mean Squared Error (RMSE) Calculations across all summary PSUR metrics for solifenacin, tadalafil, and sildenafil. PSUR Summary Statistics RMSE Total Number of Case Reports 0.58 Specified Molecule Case Reports 0.82 Unspecified Molecule Case Reports 1.00 Duplicate Case Report 0.82 MAH as a source of the report 0.58 Reported as Serious 0.58 With Concomitant Medication 1.83 Medically important conditions 0.58 This means that for all metrics on average, there was an observed difference between the manual and automated methods of 0.5 to 2 cases. The highest variation seen in the number of cases observed from the two methods is the number of unspecified molecule cases and the cases with concomitant medications. All of the remaining summary statistics had RMSE values of less than one. Therefore, all other statistics on average across all three drugs differed by less than one case. 3.2 Application of Developed Automated Process to a New Drug Product The automated process was used for the analysis of Olmesartan CV data. Olmesartan has not been run using this automated method before and therefore was run as a new drug product. There was a total of 26 cases during this reporting period, including both specified and unspecified molecule case reports. Olmesartan CV data was exported and run for the 2022-01-01 to 2022-12-31 reporting period. After running step 1 the dictionary consisted of 97 terms for user-defined expectedness analysis using the product monograph. Then step 2 was run to output a report consisting of information needed to contribute to Pharmaris’ internal database and PSUR summary information, as seen in Table 7. 35 Table 7: PSUR Summary Table for Olmesartan New Drug Run Category Count Total Number of Case Reports 26 Specified Case Reports 6 Unspecified Name Case Reports 20 Duplicate Reports 4 MAH as Source of Report 6 Reported as Serious 1 Cases with Concomitant Medications 12 Other Medically Important Conditions 1 As a result, an entire assessment of all cases, relevant information, and a summary document was all output as a result of the automated method with the user only having had to assess 97 adverse event terms and no need for any manual data entry. The complete report output of step 2 is attached in Appendix C. Next, the automated process was used to output a report for Olmesartan for the following year's reporting period. The Olmesartan CV data was run for the reporting period of 2023-01-01 to 2023-12-31 and found 15 total case reports in this reporting window, including specified and non-specified molecule cases. This would replicate a subsequent year run of the automated method. First step 1 is run, and the dictionary is updated with only new terms. Now only terms that did not exist in the previous year’s reports will need to undergo user-defined expectedness analysis. In this case, there were only 15 new adverse event terms. Then step 2 was run to output a report consisting of information needed to contribute to Pharmaris’ internal database and PSUR summary information, as seen in Table 8. Table 8: PSUR Summary Statistics for the Subsequent Year Run of Olmesartan Category Count Total Number of Case Reports 15 Specified Case Reports 7 Unspecified Case Reports 8 Duplicate Reports 2 MAH as Source of Report 2 Reported as Serious 1 Cases with Concomitant Medications 2 Other Medically Important Conditions 1 36 In the same regard as the run from the previous year mentioned above, the entire report was output with no additional need for manual data entry and the user only had to define 15 adverse event terms. Whereas, in a typical analysis for these two reporting periods a Pharmaris employee would have had to manually input and assess all 26 cases in their entirety and 15 cases for the subsequent year by looking through a PDF of over 50 pages for each reporting period. The full report and output from this subsequent year's run are attached below in Appendix C. 4. Discussion Canada Vigilance case assessment is an important part of pharmacovigilance and the tracking of AEs in Canada (5). CV is the only centralized source in Canada that tracks all AE cases reported to Health Canada (5). The efficient monitoring and analysis of this key pharmacovigilance resource is essential to keeping GVP (3). The GVP requires manufacturers to produce reports annually and keep an up-to-date internal database with all CV cases that could be attributed to their drug that Pharmaris Canada does through manual data entry (4). This is a time-consuming manual process that is prone to human error (13). Therefore, an automated method was created in this project to eliminate the need for manual data entry and simplify the process. When compared to a manual process, the developed automated process provides similar results although minor discrepancies between the approaches were noted. These discrepancies are entirely due to updates in the Canada Vigilance Database aside from the difference of cases with concomitant medications in the Sildenafil data. The manual data entry process was conducted on data retrieved between September 2023 and January 2024 depending on the drug. Unfortunately, the data used in the automated process could not be performed on these datasets as they were not in the correct format. Therefore, the data used for the automated process was from the same reporting period but was exported from the database in March 2024. Depending on the date of the export, the data can change because the CV database undergoes constant monitoring for duplicate cases or cases reported inaccurately that will be edited or changed over time (14). Additionally, when follow-up reports for a single case are received, only the latest version of the report will be included in the output (14). Therefore, as time passes reports that may have shown up in searches from 2022, may have undergone follow up and certain case characteristics could have changed (14). Upon further analysis of the datasets used for the manual method, it can be concluded that the majority of the discrepancies are due to CV having 37 updated its database. The only discrepancy that cannot be explained by the CV database itself undergoing updates is the discrepancies in the concomitant medications in the Sildenafil data. This discrepancy was the largest overall reported difference in automated and manual metrics. It was observed due to a change in the way concomitant medications are sorted in the automated method. The automated method recorded 3 fewer cases than the manual. This is because when exporting the data in an Excel format there is a column that denotes whether a drug molecule listed with a case is categorized as a suspect or concomitant drug. In the PDF version of the export, this does not exist. Therefore, when manual case assessment is done all molecules listed with a case are noted as concomitant. In the automated method, only drug products listed in the CV as concomitant are listed in the concomitant medications section of the output report. This characteristic of the export difference in a PDF format or an Excel format from a CV accounts for this discrepancy. Thus, the automated method is more accurate in only listing concomitant medications and doesn’t also list additional suspect medications with a case as concomitant. Overall, excluding cases due to updates in the CV database and the change in denoting concomitant medications, the automated process outputs exactly replicate the manual process outputs. Although the need for manual case assessment is not fully eliminated with the use of the dictionary it does decrease the time spent on user-defined expectedness analysis because once an adverse reaction term is assessed for that drug's expectedness it never needs to be assessed again. Based on the number of adverse reaction terms that are all MedDRA coded, after the first year of running the functions, the number of adverse reaction terms needing to be assessed will decrease significantly as the years progress and the dictionary is continuously updated. In the Olmesartan report creation for two consecutive years, there was a major decrease in the number of adverse event terms for manual assessment. There were in the initial run 97 adverse event terms to be assessed using the drug product monograph. In the subsequent year run, there were only 15 new adverse reaction terms. This subsequent year's dictionary update will significantly reduce the user-defined expectedness analysis burden and with continuous updates result in fewer new terms and less time spent on user input. Currently, there does exist software that automates CV case assessments as well as several other aspects of pharmacovigilance that large drug companies in Canada will subscribe to for their specific needs. For example, PVEdge offers advanced Pharmacovigilance software automation processes (15). Argus processing offers the analysis and reporting of adverse case reports (16). Cognizant offers real-time case assessment and documentation from numerous 38 sources to ensure patient safety and regulatory compliance (17). These software options are costly for company-specific solutions and are not accessible for smaller developing drug companies. As cost is a primary consideration for pharmacovigilance technology at Pharmaris, the use of open-source software, such as R, represents a viable alternative. The impact of this study on the broader research is that this automated system could provide a solution for smaller developing companies in Canada to automate their CV case assessments. The proposed method is versatile and could be used by many companies in Canada with only some small formatting adjustments to adhere to their pre-existing internal database for easy implementation. There are three major limitations associated with the proposed project; R software use in industry, manual and automated accuracy assessment not being representative of the true performance, and that reports generated are not a true representation of the frequency of AEs in Canada. Firstly, R is not widely used in industry, especially in pharmacovigilance (18). The tendency is for companies to default to paying for expensive software when R is free statistical software that is a great option for signal detection and data organization (18). As a result, there is limited knowledge of how R is used, and proper training on the automated system is required. Additionally, the continual updating and monitoring of the R software would need to occur as any changes in the data format would require the function to be modified to adhere to any changes in order to still function correctly. Secondly, due to time constraints, the accuracy assessment presented may not be representative of the true performance of the proposed automated method. The comparison between an automated method and a manual method is not representative because the manual data entry process is prone to error and is not a completely accurate standard for comparison (13). Lastly, there are limitations in the accuracy of the data used to complete these reports. Canada Vigilance is a voluntary reporting system that is prone to selection bias (19). This often results in the under-reporting of adverse events (14). Another factor that plays a role in limiting the accuracy of the number of adverse events experienced is that there is no limitation in the database to report a case (14). Cases can be added to the CV database from several sources including MAH, Hospitals, Physicians, etc. and cases do not go through further assessment before being added to the database (14). Often the same cases are reported by different sources and are not flagged as duplicate reports (14). These factors both play a role in the inaccuracy of signal detection in pharmacovigilance based on CV database reporting (19). Future directions of this project would include the automation of the expectedness analysis, the creation of training documents and resources for implementation, and a graphical user 39 interface. Although the proposed solution does eliminate the need for manual data entry, user input analysis is still required when assessing the dictionary after step 1. In the future, this step could be further automated to search the product monograph for MedDRA-coded AE terms. This would eliminate the need for any error-prone and inconsistent user-defined analysis (13). Human data entry can result in errors that could ruin or significantly skew results and conclusions drawn from the data (13). With the importance of pharmacovigilance for the safe monitoring of drug molecules, it is essential to avoid errors in data analysis to ensure the safety of drug products in Canada (3). Secondly, the creation of thorough and effective training documents is crucial for the effective use of the new automated method. There is a clear correlation between the quality of employee training and the employee's capability to perform new tasks (20). Hence, clear and effective training materials would improve the overall project implementation and would be necessary for the effective use of the automated system. Lastly, a graphical user interface could be developed to allow consumers to report AEs and for the results to be directly implemented into the Pharmaris internal database. This would also remove the necessity for manual case entry through reports received by phone, or email and reduce error (13). Overall, this project aimed to simplify and automate the Canada Vigilance case assessment process for one of our industry partners, Pharmaris Canada. This project showed that an automated process is possible, efficient, and accurate. This automated method can be used in industry to increase efficiency and eliminate manual data entry. 40 References 1. Mouchantaf R. Evolution of Pharmacovigilance in Canada with a focus on Risk Management Plans. 2012 Oct; 2. Fornasier G, Francescon S, Leone R, Baldo P. An historical overview of Pharmacovigilance. Int J Clin Pharm. 2018;40(4):744–7. 3. Pharmaris Canada. A Holistic Overview of PHARMACOVIGILANCE. 2021. 4. Canada H. Good Pharmacovigilance Practices (GVP) Guidelines (GUI-0102) [Internet]. 2006 [cited 2023 Nov 25]. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/compliance-enforcement/good-manufacturing-practices/guidance-documents/pharmacovigilance-guidelines-0102.html 5. Canada H. Canada Vigilance Program [Internet]. 2008 [cited 2023 Nov 27]. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/canada-vigilance-program.html 6. Vision for MedDRA | MedDRA [Internet]. [cited 2024 Apr 5]. Available from: https://www.meddra.org/about-meddra/vision 7. Health Canada Government of Canada [Internet]. 2009 [cited 2024 Mar 28]. Search the Canada Vigilance Adverse Reaction Online Database. Available from: https://cvp-pcv.hc-sc.gc.ca/arq-rei/index-eng.jsp 8. Product information [Internet]. [cited 2024 Apr 15]. Available from: https://health-products.canada.ca/dpd-bdpp/info?lang=eng&code=95698 9. Canada G of CIS. Appendix F - List of Drug Manufacturers [Internet]. 2019 [cited 2024 Apr 17]. Available from: https://www.sac-isc.gc.ca/eng/1573682062274/1573682081432 10. ClinSkill. Software used in Pharmacovigilance | ClinSkill Academy [Internet]. ClinSkill. 2019 [cited 2024 Apr 5]. Available from: https://www.clinskill.com/software-used-in-pharmacovigilance/ 11. Yeager K. LibGuides: Statistical & Qualitative Data Analysis Software: About R and RStudio [Internet]. 2024 [cited 2024 Mar 28]. Available from: https://libguides.library.kent.edu/statconsulting/r 12. Grolemund HW and G. 19 Functions | R for Data Science [Internet]. [cited 2024 Apr 17]. Available from: https://r4ds.had.co.nz/functions.html 13. Barchard KA, Pace LA. Preventing human error: The impact of data entry methods on data accuracy and statistical results. Comput Hum Behav. 2011 Sep 1;27(5):1834–9. 14. Canada H. Caveat, Privacy Statement and Interpretation of Data - Data Extract from the Canada Vigilance Adverse Reaction Online Database [Internet]. 2010 [cited 2024 Apr 16]. Available from: https://www.canada.ca/en/health-canada/services/drugs-health-products/medeffect-canada/adverse-reaction-database/medeffect-canada-caveat-privacy-statement-interpretation-data-extract-vigilance-adverse-reaction-online-database.html 41 15. Pharmacovigilance software; Drug Safety Database [Internet]. [cited 2024 Apr 9]. Software Used in Pharmacovigilance and Drug Safety | PvEdge. Available from: https://pvedge.sarjen.com/ 16. Argus [Internet]. [cited 2024 Apr 9]. Available from: https://www.oracle.com/ca-en/life-sciences/pharmacovigilance/argus-safety-case-management/ 17. www.cognizant.com [Internet]. [cited 2024 Apr 9]. Pharmacovigilance. Available from: https://www.cognizant.com/ca/en/industries/life-sciences-technology-solutions/pharmacovigilance-solutions 18. Using the Statistical Programming Language R in the Pharma Industry [Internet]. [cited 2024 Apr 18]. Available from: https://www.quanticate.com/blog/bid/102741/using-the-statistical-programming-language-r-in-the-pharma-industry 19. Colacci M, Fralick M. Have Current Systems of Pharmacovigilance Had Their Day? Can J Hosp Pharm. 2020;73(2):159–60. 20. Algharibeh FI, Almsafir M, Alias RB. The Relationship Between Training and Employee Performance: A Case of Jordanian Public Universities. J Adv Soc Res [Internet]. 2015 Jan 18 [cited 2024 Apr 18];4(12). Available from: https://www.sign-ific-ance.co.uk/index.php/JASR/article/view/941 42 Appendices Appendix A See the attached Google Sheets files for the output of the functions please refer to the hyperlinks below: Example Sildenafil Report Output Report Please note that the actual R output is in Excel format but was transferred to a sheet format for ease of viewing and sharing in the written thesis document. Appendix B Step 1 function: Step 1 Step 2 function: Step 2 Appendix C The outputs are located in the subfolder of the report. Both functions are in the scripts folder. This attached zip file contains in the reports subfolder both the new drug run and the subsequent year run automated report outputs of the functions in section 3.2. There is a CV export in the subfolder and step 1 and step 2 can be run for verification. Functions must be opened through the function subfolder to maintain relative paths. Step 1 can be run, the output in R should be ‘no_new_terms’ as the CV export is the same as the export used to create the 2023 Olme report. Step 2 can be run right after to verify the report compared with the pre-existing Olme 2023 report. Note after running step 2, a new file exists in the report subfolder named NEWREPORT, this is the output that the automated function has created. Olmesartan outputs and file structure can be found in the file folder attached here.
Graduating Project
eng
Unreviewed
Vancouver : University of British Columbia Library
Attribution-NonCommercial-NoDerivatives 4.0 International
Undergraduate
University of British Columbia. PHAR 491
Development of an automated process for performing pharmacovigilance case assessments
Text