UBC Faculty Research and Publications

Analytical validation of the PAM50-based Prosigna Breast Cancer Prognostic Gene Signature Assay and nCounter… Nielsen, Torsten; Wallden, Brett; Schaper, Carl; Ferree, Sean; Liu, Shuzhen; Gao, Dongxia; Barry, Garrett; Dowidar, Naeem; Maysuria, Malini; Storhoff, James Mar 13, 2014

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12885_2013_Article_4360.pdf [ 525.45kB ]
JSON: 52383-1.0223382.json
JSON-LD: 52383-1.0223382-ld.json
RDF/XML (Pretty): 52383-1.0223382-rdf.xml
RDF/JSON: 52383-1.0223382-rdf.json
Turtle: 52383-1.0223382-turtle.txt
N-Triples: 52383-1.0223382-rdf-ntriples.txt
Original Record: 52383-1.0223382-source.json
Full Text

Full Text

TECHNICAL ADVANCE Open AccessAnalytical validation of the PAM50-based ProsignaBreast Cancer Prognostic Gene Signature Assayand nCounter Analysis System using formalin-fixedparaffin-embedded breast tumor specimensTorsten Nielsen1,4*, Brett Wallden2, Carl Schaper3, Sean Ferree2, Shuzhen Liu1, Dongxia Gao1, Garrett Barry1,Naeem Dowidar2, Malini Maysuria2 and James Storhoff2AbstractBackground: NanoString’s Prosigna™ Breast Cancer Prognostic Gene Signature Assay is based on the PAM50 geneexpression signature. The test outputs a risk of recurrence (ROR) score, risk category, and intrinsic subtype (LuminalA/B, HER2-enriched, Basal-like). The studies described here were designed to validate the analytical performance ofthe test on the nCounter Analysis System across multiple laboratories.Methods: Analytical precision was measured by testing five breast tumor RNA samples across 3 sites. Reproducibilitywas measured by testing replicate tissue sections from 43 FFPE breast tumor blocks across 3 sites following independentpathology review at each site. The RNA input range was validated by comparing assay results at the extremes of thespecified range to the nominal RNA input level. Interference was evaluated by including non-tumor tissue into the test.Results: The measured standard deviation (SD) was less than 1 ROR unit within the analytical precision study and themeasured total SD was 2.9 ROR units within the reproducibility study. The ROR scores for RNA inputs at the extremesof the range were the same as those at the nominal input level. Assay results were stable in the presence of moderateamounts of surrounding non-tumor tissue (<70% by area).Conclusions: The analytical performance of NanoString’s Prosigna assay has been validated using FFPE breast tumorspecimens across multiple clinical testing laboratories.Keywords: PAM50, Analytical validation, ROR, Subtype, Breast cancer, Prosigna, NanoString, nCounter, Reproducibility,FFPE, Gene expressionBackgroundMolecular biomarkers have played an increasingly im-portant role in identifying cancer patients with differentprognostic outcomes and in predicting response tochemotherapy [1-3]. Molecular assays targeting thesebiomarkers are now routinely performed in local path-ology labs to help guide treatment decisions in breastcancer [4,5], lung cancer [6], and colorectal cancer [7].Gene expression analysis has helped identify distinctmolecular signatures in breast cancer that have differentprognostic outcomes [8-10]. Multigene assays targeting21 – 70 genes are now routinely used in clinical practiceto assess risk of recurrence in early stage breast cancer[11,12], and prospective clinical trials are also underwayto provide further supporting evidence for the clinicalutility of these assays [13,14]. To date, breast cancermultigene clinical assays have been largely limited tocentral reference laboratories due to the complexity ofperforming the test. Ultimately, development of assayswith a simplified workflow is required to move thesemultigene expression tests into the local pathology labsetting, where efficiencies such as shorter turnaround* Correspondence: torsten@mail.ubc.ca1British Columbia Cancer Agency, 3427 - 600 W 10TH Avenue, V5Z 4E6Vancouver, BC, Canada4Anatomical Pathology JPN 1401, Vancouver Hospital, 855 W. 12th Ave, V5Z1 M9 Vancouver, BC, CanadaFull list of author information is available at the end of the article© 2014 Nielsen et al.; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the CreativeCommons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, andreproduction in any medium, provided the original work is properly credited. The Creative Commons Public DomainDedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article,unless otherwise stated.Nielsen et al. BMC Cancer 2014, 14:177http://www.biomedcentral.com/1471-2407/14/177time and direct interaction between laboratory physi-cians and the clinicians will benefit active patient care.The PAM50 gene signature measures the expressionlevels of 50 genes in a surgically resected breast cancersample to classify a tumor as one of four intrinsic subtypes(Luminal A, Luminal B, HER2-enriched, and Basal-like)[15], which have been shown to be prognostic in both un-treated (i.e. no adjuvant systemic therapy) and tamoxifentreated patient populations [15,16]. In addition to identify-ing a tumor’s intrinsic subtype, the PAM50 signature gen-erates an individualized score estimating a patient’sprobability of disease recurrence by weighting the molecu-lar subtype correlations, a subset of proliferation genes,and pathologic tumor size [15,16]. The PAM50 test wasadapted to be performed using the nCounter Analysis Sys-tem in order to develop a simplified workflow that couldbe performed in a local pathology lab (Prosigna™ BreastCancer Gene Signature Assay, NanoString Technologies,Seattle). This technology uses multiplexed gene-specificfluorescently-labeled probe pairs [17] to measure gene ex-pression in frozen or formalin-fixed paraffin-embedded(FFPE) tissues with equivalent ease and efficiency [18]. Arecent clinical validation performed on RNA extractedfrom over 1000 FFPE tumor specimens from the ATACclinical trial demonstrated that the Prosigna risk of recur-rence (ROR) score, based on the PAM50 gene expressionsignature, added significant prognostic information be-yond the Oncotype DX® Recurrence Score® in estimatingthe likelihood of distant recurrence in hormone receptorpositive, post-menopausal breast cancer patients [19]treated with endocrine therapy alone. A second clinicalvalidation study performed on over 1400 FFPE patientsamples from the ABCSG-8 trial has independentlyconfirmed the clinical validity and demonstrated add-itional prognostic value in node-positive patients andfor the risk of late recurrence [20,21]. Based in parton the results from these clinical studies and the ana-lytical studies described herein, NanoString obtained aCE Mark for its Prosigna assay in 2012, followed byUS Food and Drug Administration (FDA) clearance inSeptember of 2013.Recently, requirements for demonstrating utility of atumor biomarker were established that include not onlyclinical validity, but also analytical reproducibility androbustness [22,23]. The results of ATAC and ABCSG-8,including a follow up combined analysis of the two stud-ies [24] meet this high level of evidence (Level I) forclinical validity using archived specimens [22]. The stud-ies described herein were designed to test the analyticalvalidity of decentralized use of the Prosigna assay acrossmultiple clinical testing sites, following establishedguidelines [25]. These studies were also designed to val-idate procedures for training laboratory personnel toperform the Prosigna assay on the nCounter system.MethodsNanoString Prosigna assayThe tissue input for the Prosigna assay was FFPE tissuethat had been previously diagnosed to contain viable in-vasive breast carcinoma. The breast tumor tissue mustbe classified by a pathologist as invasive carcinoma(ductal, lobular, mixed, or no special type). A pathologistreviews an H&E stain of a slide mounted tumor sectionto identify and circle the region of viable invasive breastcarcinoma. The tumor surface area on the H&E stainedsection must be ≥ 4 mm2 per slide, with tumor cellular-ity ≥ 10%. Non-tumor tissue from outside the circledarea is removed by macrodissection of the correspond-ing unstained slides. RNA was extracted from slidemounted breast tissue sections using a RNA extractionkit manufactured by Roche to NanoString’s specifica-tions [26]. For RNA isolation, a single 10-micron slidemounted tissue section was input for RNA extractionwhen the tumor surface area measured ≥ 100 mm2,whereas 3 slides were input when the tumor surfacemeasured 4-99 mm2. Following extraction of total RNAand removal of genomic DNA, RNA was eluted (30 μLvolume) and tested to ensure it met the specificationsfor concentration (≥ 12.5 ng/ μL) and purity (OD 260/280 nm 1.7-2.5).The NanoString Prosigna assay [26] measures the ex-pression levels of 50 target genes plus eight constitu-tively expressed normalization genes [15,27,28]. Assaycontrols are included to ensure that test samples and thetest process meet pre-defined quality thresholds. Ex-ogenous probes with no sequence homology to humanRNA sequences are included as positive and negativeassay controls. Positive controls are comprised of a sixpoint linear titration of in vitro transcribed RNA cover-ing an approximately 1000 fold RNA concentrationrange (0.125 – 128 fM) and corresponding probes[29,30]. Negative controls consist of a set of probes with-out the corresponding targets. Each assay run includestwo reference control samples comprising in vitro tran-scribed RNA of the 58 targets for qualification andnormalization purposes.Extracted RNA samples meeting quality and concen-tration specifications were hybridized (without reversetranscription or amplification) to capture and reporterprobes for the measured genes and assay controls. Themultiplexed hybridizations are carried out in a single-tube for 15 – 21 hrs at 65°C using 125 – 500 ng RNA(nominal input of 250 ng). After hybridization, thetarget-probe complexes were processed on the nCounterAnalysis System. Test sample data must meet a mini-mum threshold for expression of normalizing genes toensure that the assay signal is high enough for the algo-rithm to produce precise results. The linearity of thepositive control target titration and the non-specificNielsen et al. BMC Cancer 2014, 14:177 Page 2 of 14http://www.biomedcentral.com/1471-2407/14/177background from negative control probes included ineach assay is used to determine whether each assay per-formed within specification. Since the test is designed tobe run in local molecular pathology labs, all qualitythresholds are applied automatically to the data by em-bedded software; any failing metric causes an assay fail-ure notice which prevents output of a Prosigna assayresult. For samples meeting all quality thresholds, a clin-ically validated algorithm is used to determine the intrin-sic subtype and ROR score, which are prognosticindicators of risk of distant recurrence of breast cancer[19,21]. The normalized gene expression profile of eachbreast tumor sample is correlated to prototypical geneexpression profiles of the four breast cancer intrinsicsubtypes (Luminal A, Luminal B, HER2-enriched, andBasal-like). The primary tumor size (categorical inputof ≤ 2 cm or > 2 cm) and normalized gene expressionprofile of each breast tumor sample is used to calculatethe numerical ROR score. Risk categories are assigned toallow interpretation of the ROR score by using pre-specified cutoffs (defined in a clinical validation study)related to risk of distant recurrence after 10 years [19].Operators for these studies were required to undergotraining procedures to demonstrate proficiency, equiva-lent to what will be used to train users in molecularpathology laboratories for the decentralized test. Eachsite was given an overview of the NanoString technologyand Prosigna assay procedures followed by an in-lab ex-ercise where users were trained and qualified on tissueprocessing and assay procedures (requiring 10-12 hoursof total hands-on time). Briefly, each user extracted RNAfrom three FFPE breast tumor tissue samples to demon-strate proficiency in tissue processing, and each user proc-essed four prototypical breast tumor RNA samples (one ofeach intrinsic subtype with known expected ROR scorevalues) along with a negative control sample to demon-strate proficiency on the nCounter Analysis System.The analytical studies described herein were performedusing pre-specified SOPs, statistical analysis plans and ac-ceptance criteria using clinical-grade reagents, instrumen-tation, and software formatted such that no comparison ofresults between test centers could even be possible untilthe study was completed.RNA precision: study designThe RNA Precision study assessed the reproducibility ofthe Prosigna assay using a common template of purifiedRNA, thereby isolating the device-specific componentsof analytical validity from variables associated with tissueprocessing. The experimental design for analytically val-idating the precision of the assay from RNA was basedon Clinical Laboratory and Standards Institute (CLSI)guidelines for the evaluation of precision of in vitro diag-nostic devices outlined in EP05-A2 [25]. This designmeasured the variability between and within a numberof assay variables including testing site (n = 3), operator(n = 6), reagent lot (n = 3) and assay run (n = 18/site). Twoof the three sites used were CLIA-certified, CAP-accreditedlaboratories at the British Columbia Cancer Agency(Vancouver), and Washington University (St. Louis); thethird site was NanoString Technologies (Seattle).Five pooled breast tumor RNA samples were gener-ated from archived FFPE breast tumor tissue samplescontaining viable invasive breast carcinoma, to comprisea sample set representing each intrinsic breast cancersubtype and risk classification group (Table 1). Since thesamples were pooled breast tumor RNA, a default tumorsize category of ≤ 2 cm was used to determine the esti-mated ROR score, and a default nodal status of node-negative was used to determine risk category. This designensured that the prototypical gene expression profiles en-countered during routine testing were represented withinthis analytical validation study. Since Luminal subtypesmake up the vast majority of the intended use population(hormone receptor positive patients), the study designincluded three Luminal samples to span the risk classifica-tion groups. The identity of each sample aliquot was de-identified using labeled sample tubes with unique,randomly assigned, barcoded IDs to ensure that the opera-tors were blinded to any possible expected results of eachtest sample.Single use aliquots of each pooled breast tumor RNAsample and three reagent lots were distributed to eachof the three testing sites to complete the following test-ing scheme (Figure 1). Each of the five RNA pooled sam-ples was tested in duplicate during each run at thenominal RNA input level for the assay of 250 ng. Thepositions of the tumor RNA samples within the system(cartridge and strip tube position) were pre-assigned in arandomized and balanced manner for each run. Eachoperator completed one run on a given day since theassay includes an overnight hybridization step qualifyingit as a “long run method” per CLSI EP05-A2. Followinga device and study protocol familiarization run, each sitecompleted 18 valid runs (9 by each operator) (Figure 1).Table 1 RNA precision study sample summaryIntrinsicsubtypeEstimatedROR scoreRiskclassificationLuminal A 30 LowLuminal B #1 54 IntermediateLuminal B #2 64 HighBasal-like 55 IntermediateHER2-enriched 76 HighMolecular characteristics of the five pooled breast tumor RNA samples used inthe RNA precision study.Nielsen et al. BMC Cancer 2014, 14:177 Page 3 of 14http://www.biomedcentral.com/1471-2407/14/177Upon completion of the study the blinded data werecollected from each site and merged with the expectedtest result and study variables (site, operator, reagent lot,etc.) associated with each unique sample ID. The pro-spectively defined analysis plan was then executed onthe merged analysis dataset.RNA precision: statistical analysisThe pre-specified primary aim of the RNA precision val-idation was to demonstrate that there was no significantdifferences for the continuous ROR score assay outputacross the three testing sites.The following variance components model was usedto characterize the sources of variability:ROR Score = site + operator + lot + run + within-runwhere all components were treated as random compo-nents, and the RNA assay component of variation wasdefined as the sum of all these components. Variancecomponents were estimated using the R procedure“lmer”. To test whether sites were significantly different,the following versions of the above model were fitted:ROR Score = site + operator + lot + run + within-run& ROR Score = operator + lot + run + within-runwhere site was now treated as fixed and all other com-ponents were treated as random. A likelihood ratio testwith 2-degrees of freedom was performed using the fit-ted models to determine whether the effect of site wassignificant (α = 0.05). A similar analysis was performedfor the assay reagent lots.For each of the 5 pooled samples, the classificationsinto the 4 intrinsic subtype categories (Luminal A, Lu-minal B, Basal-Like, HER2-enriched) were summarizedusing frequency tables.Reproducibility: study designThe reproducibility study assessed the analytical valid-ity of the Prosigna assay, including all steps involvingin clinical lab implementation (i.e. tissue handling andRNA isolation SOPs as well as the device-specificassay steps), using a common set of breast cancer tis-sue samples.The experimental design for analytically validating thereproducibility from tissue was based on CLSI guidelinesfor the evaluation of precision of in vitro diagnostic de-vices outlined in CLSI EP05-A2. This design allows forthe measurement of variability between and within anumber of assay variables including testing site, FFPEsample block, operator, reagent lot, and assay run.A set of 43 banked FFPE breast tumor blocks fromhormone receptor positive breast cancer patients withconfirmed invasive breast carcinoma was selected fromthe biobank at Washington University at St. Louis forthis reproducibility validation study. The sample collectionand conduct of this study were conducted in compliancewith the study protocols and local IRB procedures. OneFigure 1 Overview of the design for the RNA precision validation study. Five pooled breast tumor RNA samples were tested across severalsites, operators, reagent lots, and runs.Nielsen et al. BMC Cancer 2014, 14:177 Page 4 of 14http://www.biomedcentral.com/1471-2407/14/177FFPE block for each case was selected using the followingcriteria:1. Every case should represent a unique breast cancerpatient2. All must be primary breast cancers3. All are pathology confirmed invasive ductal orlobular carcinoma, a mixtures of these types, orclassified as no special type4. All are hormone receptor positive (ER + or PgR+)breast cancer5. All must have a recorded tumor size6. FFPE blocks should be < 10 years old7. A minimum of 10 cases each of ≥ 100 mm2 tumorarea (1 slide/test) and 4 - 100 mm2 tumor area(3 slides/test)The criterion that at least 10 cases contain ≥ 100 mm2and at least 10 cases contain 4 - 99 mm2 tumor areawas implemented to validate the number of slides re-quired for the assay. The blocks were not prescreenedwith the assay prior to inclusion, but it was anticipatedthat the 43 samples would cover a broad range of RORscores representative of the intended use population,including both node-negative and node-positive pa-tients, and each risk classification group. Seventeen tis-sue samples were from node-negative patients, 6 fromnode-positive patients and 20 were from patients whoseregional lymph node status was provided by the bio-bank as NX.For reproducibility testing (Figure 2), three sets of seri-ally cut sections, each comprised of one H&E 4-micronstained slide and three 10-micron thick unstained slides,were prepared from each FFPE block. All cut and slidemounted sections were shipped to NanoString and thenone set from each of the 43 blocks was distributed to theappropriate testing site for processing. All 43 specimenswere reviewed independently by a separate pathologist foreach of the three sites.For each tissue sample, a test run consisting of macro-dissection, RNA extraction, and testing with the Pro-signa assay was performed by a single operator at eachsite following the provided standard operating proce-dures. Each operator performed a minimum of four testruns consisting of up to 10 tissue samples per run. Eachbatch of tissue samples required a minimum run time of3 days from tissue processing to result. Isolated RNAthat met the quantity and quality specifications fromeach of the slide mounted sections was tested twice inseparate assay runs. Different lots of RNA isolation kitreagents were used at each site, and a single lot of theProsigna assay kit was used at all three sites.The test results for all samples remained blinded to allpersonnel at all sites until the study was complete. Uponcompletion of the study the blinded Prosigna assay datawere collected from each site and merged with the ex-pected test result and study variables (site, operator, re-agent lot, etc.) associated with each unique sample ID.The prospectively defined analysis plan was then exe-cuted on the merged analysis dataset.Figure 2 Overview of the design for the tissue reproducibility validation study. Tissue samples (1-43) were processed in parallel acrossdifferent sites, pathologists, operators, and RNA isolation kits.Nielsen et al. BMC Cancer 2014, 14:177 Page 5 of 14http://www.biomedcentral.com/1471-2407/14/177Reproducibility: statistical analysisThe pre-specified primary aim of the tissue reproducibil-ity validation was to demonstrate the Prosigna assay ishighly reproducible, when combining all sources of vari-ation. For this study, “highly reproducible” was definedas a total standard deviation (SD) of less than 4.3 RORunits. The value of <4.3 was chosen because if two sam-ples have true ROR scores that differ by 10 units, a totalSD of 4.3 means that 95% of the time the higher of thetwo will still have a higher individual observed RORscore. A change of 10 ROR units corresponds to an aver-age change in 10-year distant recurrence free survival of7% and 6% for node negative and node positive patientsrespectively [19].The following variance components model was usedto characterize the sources of variability:Measurement = FFPE Block + site + tissue section+ errorwhere FFPE Block was treated as a fixed component,and site and section were treated as random compo-nents. The “site” term measured the systematic site-specific variation that was constant across all tissuesamples (pathologist, technician, extraction kit). The tis-sue section component measures random variation thatdiffered as a function of review/processing or withinFFPE block variation. The error term was derived fromthe duplicate RNA samples and estimated the combin-ation of run-to-run and within-run variance. Variancecomponents were estimated using the R procedure“lmer”. In the above model, the variance componentswere estimated from a combined analysis of all FFPEblocks after verifying that were no systematic changes intissue-specific variation as a function of ROR score.The tissue and RNA isolation components were esti-mated using the reproducibility validation and the assaycomponents were estimated using the RNA precisionvalidation. The total variability, σ2total , was calculated as:σ2total ¼ σ2tissue þ σ2RNAassaywhere σ2tissue was estimated as the sum of the site-to-siteand section component estimated in the tissue reprodu-cibility study, and σ2RNAassay was estimated as the totalvariation from the RNA precision study.Additional categorical analyses were performed usingtwo classifications: 3 risk-categories (low, intermediate, and high) usingboth the node-negative and node-positive cutoffs, 4 intrinsic subtype categories (Luminal A, LuminalB, Basal-Like, HER2-enriched)RNA from each tissue sample was tested twice at eachsite so there are 4 possible comparisons between sitesfor each tissue sample leading to a total number of pos-sible comparisons of 4*number of tissue samples. Foreach of the two classification schemes (risk category orsubtype), the pair-wise concordance between sites wasestimated as the fraction of all possible comparisons thatwere concordant and an exact-type 95% confidenceinterval was calculated.In addition, a post hoc analysis compared the normal-ized gene expression from the 50 classifier genes be-tween the tissue replicates from all valid specimenstested at each site using a linear regression and correl-ation analysisRNA input: study designThirteen FFPE breast tumor blocks containing pathologically-confirmed infiltrating ductal carcinoma were obtained andRNA was extracted from multiple slide mounted tissuesections from each block using the defined procedure(Figure 3). The individual RNA isolates from each FFPEblock were pooled. Each pooled tumor RNA sample wastested in duplicate across three RNA input levels withinthe assay specification range (500, 250, and 125 ng) and insinglet at two additional RNA input levels outside of thespecification range (625, 62.5 ng). Two no-target (water)measurements were also tested in duplicate on every run.All tumor RNA samples were assumed to be node-negative with a tumor size of ≤ 2 cm for this analyticalstudy since these clinical covariates have no impact on themeasured variation in the ROR score. All samples weretested using two different Prosigna assay reagent lots.RNA input: statistical analysisThe pre-specified primary aim of the RNA input studywas to demonstrate the Prosigna assay results wereFigure 3 Overview of the design for the RNA input study. RNAfrom 13 tissue samples was tested across and beyond the RNA inputrange specified for the assay.Nielsen et al. BMC Cancer 2014, 14:177 Page 6 of 14http://www.biomedcentral.com/1471-2407/14/177unchanged at the extremes of the assay specificationrange (125 and 500 ng RNA) regardless of the assay re-agent kit lot used. For each kit lot, the test statistic wasthe average difference between the mean ROR score at agiven input level RORLj and the mean ROR score atthe nominal level RORNj :Average Difference ¼ 1nXnj¼1RORLj−RORNj where the average is across the n different samples. Inthis equation, RORNj is the average of two replicates atthe nominal level and RORLj s the average of two repli-cates for input levels within specification, or is the singleresult for input levels outside of specification. Equiva-lence was pre-defined as an observed absolute averageROR difference significantly less than 3. To test the non-equivalence hypothesis that the true absolute mean dif-ference is greater than 3, a 90% confidence interval forthe difference was calculated. This 90% confidence inter-val corresponds to the two one-sided test approach forbioequivalence [31]. The input level was determined tobe equivalent to the nominal level if the 90% confidenceinterval is completely contained within -3 and 3.For each pooled sample a linear regression and correl-ation analysis was also performed between each replicateat each RNA input level and one of the two replicatesrun at 250 ng of RNA. The difference in the ROR score(ΔROR) from the nominal RNA input level (250 ng) foreach replicate at each RNA input level was calculated bysubtracting the ROR score calculated from one of thetwo replicates run at 250 ng from ROR scores calculatedat the other input levels. Additionally, the ΔROR wascalculated and linear regression and correlation analyseswere also performed between the two replicates at250 ng. The mean ΔROR, slope, intercept, and correl-ation values (with 95% confidence intervals) were calcu-lated using the pairwise comparisons for all passingsamples at each input level for both kit lots.For the no-target (water) samples, the percentage ofsamples failing the minimum threshold for expression ofnormalizing genes was calculated. All no-target sampleswere required to give a failing test result.Tissue interferents: study design and analysisTwenty three FFPE breast tumor blocks were obtainedcontaining pathologically-confirmed infiltrating ductalcarcinoma microscopically-assessed to have 10 – 95% ofthe total tissue area containing normal/non-tumor tis-sue. Pathologists identified additional tumor interferents(DCIS, necrotic tissue, or blood/hemorrhagic tissue)within or near the margins of the tumor in ten of the 23blocks.For each FFPE breast tumor block, H&E stained slideswere prepared and up to nine unstained sections werecut and mounted on slides. For the inclusion of theinterferent, the sections were processed according to theassay procedure with the exception that identified nor-mal/non-tumor tissue or any additional interferents wereincluded in the isolation (“non-macrodissected slides”).For the macrodissection where the non-tumor and otherinterferents were removed, three or (in the case of smalltumor surface areas) three and six slides were processedaccording to the Prosigna assay protocol.The change in ROR (ΔROR) due to the interferentwas calculated using the ROR score from the non-macrodissected slides minus the ROR score from themacrodissected slides (Figure 4). For the tissue blockswhere three and six macrodissected slides were inde-pendently isolated and both produced a passing assay re-sult, the average of the two ROR scores were used tocalculate the ΔROR.ResultsRNA precision: variance components analysisThe precision of the Prosigna assay starting from RNA wasassessed with 5 pooled breast tumor RNA samples eachtested 36 times at each of the three sites. There were no in-dividual test samples that failed the pre-specified data QCmetrics in the software so the analysis includes 540 resultsfrom 54 valid runs. For all five tumor RNA samples, thetotal SD was less than 1 ROR unit on a 0 - 100 scale(Table 2), and there was 100% concordance between mea-sured subtype result and expected subtype result as well asmeasured and expected risk group. More than 60% ofthe measured variability came from within-run variance(repeatability) while less than 2% of the variance wasattributable to site-to-site variance or operator-to-operatorvariance. The differences in mean ROR scores between siteswere less than 0.5 ROR units on a 0-100 scale and wereinsignificant for all tested samples (Additional file 1:Table S1). The contribution to overall variance by the threereagent lots was approximately 20% of the total variance onaverage, but the differences were all less than 1 ROR unit.At each site, the normalized gene expression between RNAreplicates was highly correlated with slopes ranging from0.98 – 1.00, intercepts at 0, and r values of 0.99.The distribution of measured ROR scores for each ofthe five pooled RNA samples was also examined acrossthe three lots, six users and three test sites. The range ofROR scores for the 108 independent measurements was≤4 units for each of the 5 sample pools (Figure 5).Reproducibility: test sample quality control andcharacterizationThe call rate for the 43 tissue samples evaluated was95%, 93%, and 100% for sites 1, 2, and 3 respectively.Nielsen et al. BMC Cancer 2014, 14:177 Page 7 of 14http://www.biomedcentral.com/1471-2407/14/177Forty samples yielded results at all sites (RNA isolationof one sample at one site required repeating). One tissuesample yielded results at 2 sites, and 2 samples yieldedresults at a single site, while the other sites did not ob-tain sufficient RNA to perform the assay for these sam-ples. The measured tumor surface area for 4/5 RNAisolation failures was very small (≤ 15 mm2). One hun-dred percent (100%) of samples passing tissue reviewand RNA isolation specifications yielded passing resultsfrom the Prosigna assay.The calculated test results from the 43 tissuesacross all sites represent a wide range (94 units) ofROR scores (Figure 6) and all risk categories whenapplying the node-negative or node-positive RORscore cutoffs to all samples. All four intrinsic subtypeswere also represented among the 43 specimens. Thetwo samples where RNA could only be successfullyisolated at one site were excluded from all subsequentstatistical analysis as there was no available data forcomparing across sites. Both of these samples hadROR scores of less than 10 and were classified asLuminal A.Reproducibility: variance components analysis(primary objective)Table 3 shows the results of the variance components ana-lysis using all 41 tissue specimens where replicate measure-ments were available. The “tissue section” variation, whichconsists of variation contributed by within FFPE block sec-tions, pathology review, and tissue processing, was thedominant source of variation (> 90% of total variance).The differences on average between the sites were negligible(< 1% of total variance). The combined run-to-run variabil-ity and within-run variability in the assay (determined fromthe duplicate measurements from each RNA isolation fromthe reproducibility study) was consistent with the variabilitymeasured in the RNA-precision study (variance of 0.51compared to 0.47 for the RNA-precision study).The total SD including all source of variation (tissueand RNA processing variability) was 2.9 indicating thatFigure 4 Overview of tissue processing for assessing the effect of tissue interferents. Multiple sections from FFPE breast tumor blocks weremounted onto slides and processed with or without macrodissection. The change in ROR score (ΔROR) is calculated as the ROR score from thenon-macrodissected slides minus the ROR score from the macrodissected slides (or in the illustration ΔROR = 25 – 30 = -5).Table 2 Variance components for the five pooled RNA samples across 108 replicatesPooled RNAsampleMean RORscoreVariance component (%) TotalvarianceTotalSDReagent lot Site Operator Run Within-runBasal-like 55.4 0.059 (20%) 0.000 (0%) 0.000 (0%) 0.046 (15%) 0.194 (65%) 0.299 (100%) 0.55HER2-enriched 76.2 0.165 (37%) 0.000 (0%) 0.000 (0%) 0.000 (0%) 0.277 (63%) 0.442 (100%) 0.66Luminal A 31.4 0.010 (2%) 0.000 (0%) 0.000 (0%) 0.134 (30%) 0.296 (67%) 0.44 (100%) 0.66Luminal B 1 55.0 0.105 (18%) 0.000 (0%) 0.000 (0%) 0.046 (8%) 0.426 (74%) 0.576 (100%) 0.76Luminal B 2 64.8 0.119 (21%) 0.014 (2%) 0.000 (0%) 0.064 (11%) 0.380 (66%) 0.576 (100%) 0.76The percent of total variance is listed below the estimated variance.Nielsen et al. BMC Cancer 2014, 14:177 Page 8 of 14http://www.biomedcentral.com/1471-2407/14/177the Prosigna assay can measure a difference between twoROR scores of 6.75 with 95% confidence.Reproducibility: subtype and risk category classificationsconcordanceThe site-to-site concordances for the two categoricalclassifications are shown in Table 4, in each case withexact-type 95% confidence intervals. For each compari-son (subtype and node negative and positive risk cat-egories), the average concordance between sites was atleast 90%. There were no samples where the risk cat-egory changed from low risk to high risk (or vice versa)between or within sites when the samples were assumedto be from node negative patients. There were only twointermediate/high risk samples that did not give identicalsubtypes across all 6 replicates: One sample had duplicate Luminal A results at onesite and duplicate Luminal B results at each of theother two sites. One specimen had duplicate Luminal A results atone site, duplicate HER2-enriched results at anothersite and one each of Luminal A and HER2-enrichedat the third site.Reproducibility: pairwise correlation coefficients of geneexpressionThe average intercept, slope, and Pearson’s correlationof the pair-wise comparisons between sites are reportedFigure 5 Distribution of 108 ROR scores measured for each of the 5 Pooled RNA samples. Boxplots show the distribution of ROR scoresrelative to the 0-100 range and the histograms show the frequency of the measured ROR scores on a 20-point range. Boxplots and histogramsare colored by the intrinsic subtype result for each sample.Figure 6 Reproducibility of the ROR score in the tissuereproducibility study. Average tissue block ROR compared to theindividual ROR score for all samples. Data are colored by the intrinsicsubtype result. The high, intermediate, and low node negative riskcategories are shown to the right of the figure with the risk thresholdsshown as lines in the body of the figure.Table 3 Total variability (from tissue and RNA processing)of the Prosigna assayTissue processing variability RNA processingvariabilityTotalvariabilityTotalSDSite Within block/process0.10 7.72 0.47 8.29 2.9The total SD of 2.9 is on a 0-100 ROR scale.Nielsen et al. BMC Cancer 2014, 14:177 Page 9 of 14http://www.biomedcentral.com/1471-2407/14/177with the 95% confidence interval (Table 5). The gene ex-pression between tissue replicates was highly correlatedbetween sites with slopes ranging from 0.97 – 1.00, in-tercepts at 0, and r values of 0.98 or greater. Equivalentor higher correlation values were observed when a simi-lar analysis was performed for the RNA replicates testedat each site (Additional file 2: Table S2). Additionally,hierarchical clustering analysis demonstrated that tissuesample and RNA sample replicates were always and onlyclustered together across a wide range of expression ineach of the 50 genes across all samples tested (Additionalfile 3: Figure S1).RNA input: test sample quality controlThe average ROR score for the tested samples covered abroad range (20 – 82) and all intrinsic subtypes – in-cluding 5 Luminal A, 4 Luminal B, 3 HER2-enrichedand 1 Basal-like sample (Additional file 4: FigureS2).One FFPE block was tested with a single kit lot due toinsufficient RNA mass from the isolation for the secondlot. Two runs (each with different samples) failed to pro-vide passing results for one of the two lots tested due toa processing error detected by system controls withinsufficient RNA to repeat the assay. All measured no-target samples (n = 46) were well below the threshold forsignal and yielded a failing test result (0% call rate). Alltumor RNA measurements within assay specification (n =138) yielded a passing test result (100% call rate). Onehundred percent (100%) of specimens with input abovespecification (625 ng) yielded a passing test result. Eighty-three percent (83%) of specimens (10/12) tested at inputbelow specification (62.5 ng) yielded a test result in lot 1,as did 100% in lot 2.RNA input: ROR score difference and pairwise correlationcoefficients of gene expressionFor each of the two reagent lots tested, the confidenceinterval around the mean ROR score difference betweenthe nominal input and the RNA input limits (125 and500 ng) were completely contained within -3 and 3 RORunits. The ROR scores at 125 and 500 ng RNA weretherefore equivalent to those at the target input concen-tration of 250 ng for each of the two reagent kit lots testedmeeting the primary objective of the study. Of note, whencharacterizing the RNA levels outside of the assay specifi-cation, the ROR scores at 62.5 ng RNA were not equiva-lent (with an upper confidence limit at 3.26) to those atthe target input concentration of 250 ng for one of thetwo lots tested. This illustrates the importance of perform-ing the assay according to the defined procedure.When the lots were combined the normalized gene ex-pression values and ROR scores were consistent to thoseat the target input concentration of 250 ng within andeven outside the RNA input limit specifications (Table 6).Characterization of intrinsic subtype across the samplestested shows a 100% concordance in subtype call acrossall samples and inputs. Similarly, there is a 100% concord-ance by risk classification across all samples and inputs.Tissue interferents: test sample quality controlOut of 23 samples six were Luminal A, seven were Lu-minal B, two were HER2-enriched, and eight were Basal-like. The average ROR score for the 23 samples covered abroad range (10 – 83), (Additional file 5: Figure S3).Table 4 Concordance of subtype calls and risk categories between the three sitesComparisontypePairwise Concordance [95% CI] AverageconcordanceSite 1 vs. Site 2 Site 1 vs. Site 3 Site 2 vs. Site 3(n = 40) (n = 41) (n = 40)Subtype 96.3% 98.8% 95% 97%[86.4%–99.5%] [91.0%–100%] [83.1%–99.3%]Risk Category 87.5% 92.7% 90% 90%(Node Negative) [73.2%–95.8%] [80.1%–98.4%] [76.4%–97.2%]Risk Category 90.0% 95.1% 95.0% 93%(Node Positive) [76.9%–96.0%] [83.9%–98.7%] [83.5%–98.6%]The pairwise (site to site) concordance is reported with the 95% confidence interval.Table 5 Site to site gene expression comparisons fromthe tissue reproducibility studyComparisonPairwise (n) Intercept Slope Pearson[95% CI] [95% CI] [95% CI]All Sites 121 0.00 0.98 0.98[-0.01–0.01] [0.97–0.99] [0.98–0.98]Site 1 vs. Site 2 40 0.00 0.97 0.98[-0.01–0.01] [0.95–0.98] [0.97–0.98]Site 1 vs. Site 3 40 0.01 1.00 0.98[0–0.02] [0.98–1.01] [0.98–0.99]Site 2 vs. Site 3 41 -0.01 0.99 0.99[-0.02–0] [0.97–1] [0.98–0.99]Pairwise correlations, slopes, and intercepts of normalized 50 genes for tissuesreplicates from the tissue reproducibility study. The average intercept, slope,and Pearson’s correlation of the pair-wise comparisons are reported with their95% confidence intervals.Nielsen et al. BMC Cancer 2014, 14:177 Page 10 of 14http://www.biomedcentral.com/1471-2407/14/177Tissue interferents: impact on ROR scoreAs the amount of adjacent non-tumor tissue increasesthere is an increasing risk that the reported ROR scorewill be an underestimate or negatively biased (up to -19ROR score units for samples containing 95% non-tumortissue) estimate of a patient’s risk of recurrence (Figure 7).Elimination of the macrodissection step required by theassay also caused a change in subtype determination forfive out of 23 samples. Three Luminal B samples, oneHER2-enriched, and one basal-like sample were classifiedas Luminal A due to inclusion of adjacent non-tumor tis-sue. In contrast, the presence of intratumor hemorrhage,necrosis or DCIS (not removed by macrodissection) hadlittle effect on ROR.DiscussionBreast cancer gene expression testing has been the sub-ject of many studies demonstrating its capacity to strat-ify breast cancers by prognostic risk [9,15,16,32,33].Increasingly, studies are also showing the value of suchsignatures to predict response to therapy, for example byusing these tests to evaluate archival specimens fromrandomized clinical trials [34-36]. The integration ofmolecular genomic testing into cancer care is an activearea of development, with huge genomic datasets be-coming available. Great improvements in experimentaldesign and bioinformatic analysis have led to the devel-opment of robust signatures ripe for translation intoclinical tests. Studies applying these signatures to differ-ent clinical series with observational, case-control, co-hort and randomized trial designs have generatedincreasingly strong evidence for clinical validity, particu-larly in breast cancer [19,34,35,37]. It is in this backdropthat the Evaluation of Genomic Applications in Practiceand Prevention (EGAPP) working group was formed toguide best practices in experimental design and the in-terpretation of evidence for utility in clinical practice[23]. Fundamental to EGAPP criteria is the concept thatclinical utility requires not only clinical validity (linkingtest results to clinical presentation, treatment and out-come), but just as importantly, analytical validity (thecapacity of the test classifier to be sensitive, specific andreproducible in practice). However, EGAPP found thatrelatively few studies of breast cancer molecular classifiershave directly reported on analytical reproducibility [38].Analytical reproducibility is a requirement for the im-plementation of all diagnostic tests, but it is especiallycritical for decentralized tests given the challenges ofmaintaining reproducibility across pathologists, technicaloperators, and instrumentation. However, decentralizedtests also have many advantages over Laboratory Devel-oped Tests that are performed at single central labora-tories. By avoiding the need for shipping tissues,turnaround times and costs are reduced. The capacityfor the laboratory physician to interact directly with thetreating physician greatly aids medical care, for example infacilitating appropriate prioritization of critical specimens,Table 6 Comparison of gene expression at different masses from the RNA input studyMass (ng) Pairwise (n) Pearson Slope Intercept ΔROR[95% CI] [95% CI] [95% CI] [95% CI]62.5 21 0.97 [0.93–0.99] 0.96 [0.91–1.00] -0.02 [-0.05–0.01] 0.48 [-1.27–2.22]125 46 0.99 [0.97–0.99] 0.98 [0.96–1.01] -0.01 [-0.03–0.01] -0.04 [-0.89–0.8]250 23 0.99 [0.98–1.00] 1.00 [0.99–1.01] 0.00 [-0.01–0.01] -0.39 [-0.96–0.17]500 46 0.99 [0.99–1.00] 0.97 [0.96–0.99] 0.02 [0.01–0.04] -0.57 [-1.39–0.26]625 23 0.99 [0.98–1.00] 0.95 [0.92–0.99] 0.03 [0.01–0.06] -0.78 [-2.2–0.63]Pairwise correlations, slopes, and intercepts of normalized 50 genes and change in ROR score for replicate RNA Hybridizations with different mass inputs. Theaverage intercept, slope, Pearson’s correlation, and change in ROR for the pair-wise comparisons are reported with their 95% confidence intervals.Figure 7 Effect of non-tumor tissue on the ROR score. Theimpact of including adjacent non-tumor tissue on ROR was assessedby determining the change in test results from slide mountedsections with vs. without macrodissection of adjacent non-tumortissue. Data colors represent if the interferent was only normal/non-tumor tissue or if additional non-tumor interferents (DCIS, necrotictissue, or blood/hemorrhagic tissue) were identified within or nearthe margins of the tumor.Nielsen et al. BMC Cancer 2014, 14:177 Page 11 of 14http://www.biomedcentral.com/1471-2407/14/177explaining equivocal or unexpected results, and quicklyrecognizing inadequate specimens and what can be doneto get a result helpful to the patient as soon as possible.Although first generation breast cancer prognostic testswere performed in central labs [32], second generationtests are being developed and validated to realize the ad-vantages of decentralized testing [39].The Prosigna assay was tested across a range of RNAmass inputs that is consistent with what will be expectedin a clinical lab setting. The assay is robust across thatrange, similar to what has been reported with other mul-tigene breast cancer tests [32,39]. Additionally, the assaygave consistent results outside the specified assay RNAinput limits; only 2 samples failed to produce passing re-sults at half the lowest specified mass further illustratingthe robustness of the assay.The observation of biased subtype calls and ROR scoreswith the inclusion of non-tumor tissue is consistent with aprior study [40], however the bias reported herein is lesssevere. Similar to what is expected to be experienced inclinical practice, the interferent being measured here is re-ported as percent adjacent non-tumor tissue included, ra-ther than percent non-tumor RNA from a separate pairednormal tissue sample reported in the earlier study. Normalbreast tissue yields less total RNA compared to tumor tis-sue [41] and adjacent non-tumor tissue at the margins ofthe tumor have certain cancer pathways activated wherematched healthy breast does not [42,43]. Nonetheless, thisstudy illustrates the importance of performing themacrodissection according to the defined procedure tomaximize the accuracy of the test.The precision and reproducibility of the Prosignaassay, estimated from repeat measurements of pooledtumor RNA sample(s) and de-identified patient tissuesamples across multiple testing sites is similar (relativeto the overall test range) to what was previously re-ported for centralized lab tests [32,38]. These resultsdemonstrate that the Prosigna assay is analytically repro-ducible even when performed at multiple test sites andincluding all process variables. It will be important forlocal labs to verify the reproducibility reported hereinwhen implementing this decentralized assay to ensure thequality of the test’s results, including ongoing processmonitoring [38].Our experience of implementing the nCounter plat-form in our CLIA-certified hospital laboratory environ-ments proved to be straightforward, confirming thesimplicity of the assay and its suitability as an in vitrodiagnostic test. Training of the assay workflow (includ-ing tissue macrodissection, RNA isolation and setup ofProsigna assay) takes less than one week. The pre-specified SOPs are easy to follow and the procedure ofRNA extraction and Prosigna assay are straightforward.All operators, most of whom were naïve users to theProsigna assay, were able to pass the training procedureson the first attempt, before executing the pre-specifiedstudy protocols. Although overnight incubations are re-quired during RNA extraction and RNA – probehybridization, the incubation temperature is constant, andhands-on time requirements for the whole experiment arevery limited. Furthermore, the analyses for subtype calland ROR score are simplified and controlled by integrat-ing the algorithm into the software for raw data process-ing, reducing the potential for human error in datacleaning and analysis.ConclusionThe FDA cleared and CE marked Prosigna assay based onthe PAM50 gene expression signature has recently beenshown to predict the risk of distant recurrence in womenwith hormone receptor positive early stage breast cancertreated with five years of endocrine therapy [19,20]. Thisdemonstration of analytical reproducibility generates astrong body of evidence supporting the decentralized useof this test as a tool for breast cancer risk stratification.Additional ongoing studies of the clinical validity of thePAM50 gene expression signature for chemosensitivityprediction [34-36] could, if confirmed, be considered clin-ically actionable given the demonstrated analytical validityof this test.Additional filesAdditional file 1: Table S1. Site to site ROR sample means. Mean RORscores were calculated for each pooled RNA sample, and likelihood ratiotest for significance was performed to test for differences between sites.There were no significant differences in the results observed across sitesfor the five pooled RNA samples tested. All p-values were well above 0.05for the likelihood ratio test of significance of site with 2 degrees offreedom for each pooled RNA sample. The differences in means betweensites were all less than 0.5 ROR units on a 0-100 scale.Additional file 2: Table S2. Within site gene expression comparisonsfrom the tissue reproducibility study. Pairwise correlations, slopes, andintercepts of normalized 50 genes for replicate RNA Hybridizations fromthe tissue reproducibility study. The average intercept, slope, andPearson’s correlation of the pair-wise comparisons are reported with the95% confidence interval.Additional file 3: Figure S1. Hierarchical clustering of all samples fromthe tissue reproducibility study. Clustering analysis (using a Pearson’sdistance metric and average linkage) was performed on the mediancentered normalized, Log2 transformed and scaled sample data tofurther characterize the gene expression in the tissue samples. The tissuesample and RNA sample replicates were always only clustered togetherand the node heights are almost imperceptibly low (indicating highlycorrelated gene expression).Additional file 4: Figure S2. Average ROR Score for the 13 uniquetumor RNA samples within the RNA Input Study. Data are colored by theintrinsic subtype result at 250 ng of RNA.Additional file 5: Figure S3. ROR Score for the 23 uniquemacrodissected tumor samples. Data are colored by the intrinsic subtyperesult for each tissue. For tissues with multiple isolations the subtype resultillustrated was from the macrodissection with the most number of slidesprocessed.Nielsen et al. BMC Cancer 2014, 14:177 Page 12 of 14http://www.biomedcentral.com/1471-2407/14/177AbbreviationsROR: Risk of recurrence; SD: Standard deviation; FDA: US Food and DrugAdministration; CLSI: Clinical Laboratory and Standards Institute;EGAPP: Evaluation of genomic applications in practice and prevention;FFPE: Formalin-fixed paraffin-embedded.Competing interestsTN disclosed that he is one of the holders of the patents on which theProsigna Assay is based and is a co-founder of Bioclassifier, LLC whichlicenses the PAM50 algorithm to NanoString Technologies, Inc. JS, SF, BW,ND, and MM all disclose that they are employees of and shareholders inNanoString Technologies. CS is a paid consultant of NanoString Technologies,Inc. All other authors had no disclosures to report.Author’s contributionsTN, JS, BW, CS, SF, SL contributed to the study design and protocols anddrafted the manuscript. CS, JS, and BW performed statistical analysis andpresentation of data. DG performed tissue review for the tissuereproducibility study and the tissue interference study. ND, MM, and GBperformed the assay for the RNA precision study. ND and GB performed theassay for the tissue reproducibility study. ND performed the assay for theRNA input and tissue interferents studies. All authors read and approved thefinal manuscript.AcknowledgmentsWe thank Sandra McDonald, MD from Washington University School ofMedicine for her excellent work in procuring, maintaining the integrity andde-identification, and histological review of tissue specimens for the tissuereproducibility study.We thank Shashikant Kulkarni PhD, FACMG, Vishwanathan HucthagowderPhD, and Mike Evenson for providing laboratory space, technical expertise,and execution of tissue reproducibility and RNA precision protocols.We thank Katherine Deschryver, MD from Washington University School ofMedicine for her histological review of tissue specimens for the tissuereproducibility study.We thank Nasrin Mawji from the Centre for Translational and AppliedGenomics at BC Cancer Agency for her work in executing the RNA precisionprotocol.Author details1British Columbia Cancer Agency, 3427 - 600 W 10TH Avenue, V5Z 4E6Vancouver, BC, Canada. 2NanoString Technologies, Inc., 530 Fairview AvenueNorth, Suite 2000, Seattle, WA, USA. 3Myraqa, 3 Lagoon Drive, RedwoodShores, CA, USA. 4Anatomical Pathology JPN 1401, Vancouver Hospital, 855W. 12th Ave, V5Z 1 M9 Vancouver, BC, Canada.Received: 24 October 2013 Accepted: 12 February 2014Published: 13 March 2014References1. Henry NL, Hayes DF: Cancer biomarkers. Mol Oncol 2012, 6(2):140–146.2. Dietel M, Johrens K, Laffert M, Hummel M, Blaker H, Muller BM, Lehmann A,Denkert C, Heppner FL, Koch A, Sers C, Anagnostopoulos I: Predictivemolecular pathology and its role in targeted cancer therapy: a reviewfocussing on clinical relevance. Cancer Gene Ther 2013, 20(4):211–221.3. Duffy MJ, Crown J: Companion Biomarkers: Paving the Pathway toPersonalized Treatment for Cancer. Clin Chem 2013, 59(10):1447–1456.4. Gown AM: Current issues in ER and HER2 testing by IHC in breast cancer.Mod Pathol 2008, 21(Suppl 2):S8–S15.5. Wolff AC, Hammond ME, Schwartz JN, Hagerty KL, Allred DC, Cote RJ,Dowsett M, Fitzgibbons PL, Hanna WM, Langer A, McShane LM, Paik S,Pegram MD, Perez EA, Press MF, Rhodes A, Sturgeon C, Taube SE, Tubbs R,Vance GH, van de Vijver M, Wheeler TM, Hayes DF, American Society ofClinical Oncology, College of American Pathologists: American Society ofClinical Oncology/College of American Pathologists guidelinerecommendations for human epidermal growth factor receptor 2 testingin breast cancer. J Clin Oncol 2007, 25(1):118–145.6. Sholl LM, Xiao Y, Joshi V, Yeap BY, Cioffredi LA, Jackman DM, Lee C, JannePA, Lindeman NI: EGFR mutation is a better predictor of response totyrosine kinase inhibitors in non-small cell lung carcinoma than FISH,CISH, and immunohistochemistry. Am J Clin Pathol 2010, 133(6):922–934.7. Weichert W, Schewe C, Lehmann A, Sers C, Denkert C, Budczies J,Stenzinger A, Joos H, Landt O, Heiser V, Rocken C, Dietel M: KRASgenotyping of paraffin-embedded colorectal cancer tissue in routinediagnostics: comparison of methods and impact of histology. J MolDiagn 2010, 12(1):35–42.8. Perou CM, Sorlie T, Eisen MB, van de Rijn M, Jeffrey SS, Rees CA, Pollack JR,Ross DT, Johnsen H, Akslen LA, Fluge O, Pergamenschikov A, Williams C,Zhu SX, Lonning PE, Borresen-Dale AL, Brown PO, Botstein D: Molecularportraits of human breast tumours. Nature 2000, 406(6797):747–752.9. Paik S, Shak S, Tang G, Kim C, Baker J, Cronin M, Baehner FL, Walker MG,Watson D, Park T, Hiller W, Fisher ER, Wickerham DL, Bryant J, Wolmark N: Amultigene assay to predict recurrence of tamoxifen-treated, node-negative breast cancer. N Engl J Med 2004, 351(27):2817–2826.10. van de Vijver MJ, He YD, van’t Veer LJ, Dai H, Hart AA, Voskuil DW, SchreiberGJ, Peterse JL, Roberts C, Marton MJ, Parrish M, Atsma D, Witteveen A, GlasA, Delahaye L, van der Velde T, Bartelink H, Rodenhuis S, Rutgers ET, FriendSH, Bernards R: A gene-expression signature as a predictor of survival inbreast cancer. N Engl J Med 2002, 347(25):1999–2009.11. Arpino G, Generali D, Sapino A, Del Lucia M, Frassoldati A, De Laurentis M,Paolo P, Mustacchi G, Cazzaniga M, De Placido S, Conte P, Cappelletti M,Zanoni V, Antonelli A, Martinotti M, Puglisi F, Berruti A, Bottini A, Dogliotti L:Gene expression profiling in breast cancer: a clinical perspective.Breast 2013, 22(2):109–120.12. Harris L, Fritsche H, Mennel R, Norton L, Ravdin P, Taube S, Somerfield MR,Hayes DF, Bast RC Jr, American Society of Clinical Oncology: American Societyof Clinical Oncology 2007 update of recommendations for the use oftumor markers in breast cancer. J Clin Oncol 2007, 25(33):5287–5312.13. Zujewski JA, Kamin L: Trial assessing individualized options for treatmentfor breast cancer: the TAILORx trial. Future Oncol 2008, 4(5):603–610.14. Cardoso F, Van’t Veer L, Rutgers E, Loi S, Mook S, Piccart-Gebhart MJ: Clinicalapplication of the 70-gene profile: the MINDACT trial. J Clin Oncol 2008,26(5):729–735.15. Parker JS, Mullins M, Cheang MC, Leung S, Voduc D, Vickery T, Davies S,Fauron C, He X, Hu Z, Quackenbush JF, Stijleman IJ, Palazzo J, Marron JS,Nobel AB, Mardis E, Nielsen TO, Ellis MJ, Perou CM, Bernard PS: Supervisedrisk predictor of breast cancer based on intrinsic subtypes. J Clin Oncol2009, 27(8):1160–1167.16. Nielsen TO, Parker JS, Leung S, Voduc D, Ebbert M, Vickery T, Davies SR, SniderJ, Stijleman IJ, Reed J, Cheang MC, Mardis ER, Perou CM, Bernard PS, Ellis MJ: Acomparison of PAM50 intrinsic subtyping with immunohistochemistry andclinical prognostic factors in tamoxifen-treated estrogen receptor-positivebreast cancer. Clin Cancer Res 2010, 16(21):5222–5232.17. Geiss GK, Bumgarner RE, Birditt B, Dahl T, Dowidar N, Dunaway DL, Fell HP,Ferree S, George RD, Grogan T, James JJ, Maysuria M, Mitton JD, Oliveri P,Osborn JL, Peng T, Ratcliffe AL, Webster PJ, Davidson EH, Hood L, DimitrovK: Direct multiplexed measurement of gene expression with color-codedprobe pairs. Nat Biotechnol 2008, 26(3):317–325.18. Reis PP, Waldron L, Goswami RS, Xu W, Xuan Y, Perez-Ordonez B, Gullane P,Irish J, Jurisica I, Kamel-Reid S: mRNA transcript quantification in archivalsamples using multiplexed, color-coded probes. BMC Biotechnol 2011,11:46–6750. 11-46.19. Dowsett M, Sestak I, Lopez-Knowles E, Sidhu K, Dunbier AK, Cowens JW,Ferree S, Storhoff J, Schaper C, Cuzick J: Comparison of PAM50 Risk ofRecurrence Score With Oncotype DX and IHC4 for Predicting Risk ofDistant Recurrence After Endocrine Therapy. J Clin Oncol 2013,31(22):2783–2790.20. Gnant M, Filipits M, Mlineritsch B, Dubsky P, Jakesz R, Kwasny W, Fitzal F,Rudas M, Knauer M, Singer C, Greil R, Ferree S, Storhoff J, Cowens J, SchaperC, Liu S, Nielsen T: Clinical validation of the PAM50 risk of recurrence(ROR) score for predicting residual risk of distant-recurrence (DR) afterendocrine therapy in postmenopausal women with ER + early breastcancer (EBC): An ABCSG study [abstract]. Cancer Res 2012, 72(24):P2–10-02.21. Gnant M, Filipits M, Dubsky P, Rudas M, Balic M, Greil R, Ferree S, Cowens J,Schaper C, Nielsen T: 53O_PR Predicting risk for late metastasis: ThePAM50 risk of recurrence (ROR) score after 5 years of endocrine therapyin postmenopausal women with HR + early breast cancer: A study on1,478 patients from the ABCSG-8 trial [abstract]. Ann Oncol 2013, 24(3):29.doi:10.1093/annonc/mdt084.1.22. Simon RM, Paik S, Hayes DF: Use of archived specimens in evaluation ofprognostic and predictive biomarkers. J Natl Cancer Inst 2009,101(21):1446–1452.Nielsen et al. BMC Cancer 2014, 14:177 Page 13 of 14http://www.biomedcentral.com/1471-2407/14/17723. Teutsch SM, Bradley LA, Palomaki GE, Haddow JE, Piper M, Calonge N,Dotson WD, Douglas MP, Berg AO, EGAPP Working Group: The Evaluationof Genomic Applications in Practice and Prevention (EGAPP) Initiative:methods of the EGAPP Working Group. Genet Med 2009, 11(1):3–14.24. Gnant M, Dowsett M, Filipits M, Lopez-Knowles E, Greil R, Balic M, Cowens J,Nielsen T, Shaper C, Sestak I, Fesl C, Cuzick J: Identifying clinically relevantprognostic subgroups in node-positive postmenopausal HR + earlybreast cancer patients treated with endocrine therapy: A combinedanalysis of 2,485 patients from ABCSG-8 and ATAC using the PAM50 riskof recurrence (ROR) score and intrinsic subtype [abstract]. J Clin Oncol2013, 31:506.25. Tholen D, Kallner A, Kennedy J, Krouwer J, Meier K: Clinical and LaboratoryStandards Institute (CLSI). Evaluation of precision performance of quantitativemeasurement methods; approved guideline – second edition. CLSI documentEP05-A2. 950 west valley road, suite 2500. Wayne, PA, USA: CLSI; 2004.26. NanoString Technologies Inc: Prosigna™ Breast Cancer Prognostic Gene SignatureAssay [Package Insert]. Seattle, WA: NanoString Technologies, Inc; 2013.27. Majidzadeh-A K, Esmaeili R, Abdoli N: TFRC and ACTB as the bestreference genes to quantify Urokinase Plasminogen Activator in breastcancer. BMC Res Notes 2011, 4:215–0500. 4-215.28. Szabo A, Perou CM, Karaca M, Perreard L, Palais R, Quackenbush JF, BernardPS: Statistical modeling for selecting housekeeper genes. Genome Biol2004, 5(8):R59.29. Baker SC, Bauer SR, Beyer RP, Brenton JD, Bromley B, Burrill J, Causton H,Conley MP, Elespuru R, Fero M, Foy C, Fuscoe J, Gao X, Gerhold DL, Gilles P,Goodsaid F, Guo X, Hackett J, Hockett RD, Ikonomi P, Irizarry RA, KawasakiES, Kaysser-Kranich T, Kerr K, Kiser G, Koch WH, Lee KY, Liu C, Liu ZL, LucasA, Manohar CF, Miyada G, Modrusan Z, Parkes H, Puri RK, Reid L, Ryder TB,Salit M, Samaha RR, Scherf U, Sendera TJ, Setterquist RA, Shi L, Shippy R,Soriano JV, Wagar EA, Warrington JA, Williams M, Wilmer F, Wilson M, WolberPK, Wu X, Zadro R, External RNA Controls Consortium: The External RNAControls Consortium: a progress report. Nat Methods 2005, 2(10):731–734.30. Warrington J, Corbisier P, Feilotter H, Hackett J, Reid L, Salit M, Wagar E,Williams P, Wolber P: Clinical and Laboratory Standards Institute (CLSI). Use ofExternal RNA Controls in Gene Expression Assays; Approved Guideline. CLSIDocument MM16-A. 940 west valley road, suite 1400. Wayne, PA, USA: CLSI;2006.31. Berger R, Hsu J: Bioequivalence trials, intersection-union tests andequivalence confidence sets. Stat Sci 1996, 11:283–319.32. Cronin M, Sangli C, Liu ML, Pho M, Dutta D, Nguyen A, Jeong J, Wu J,Langone KC, Watson D: Analytical validation of the Oncotype DXgenomic diagnostic test for recurrence prognosis and therapeuticresponse prediction in node-negative, estrogen receptor-positive breastcancer. Clin Chem 2007, 53(6):1084–1091.33. Nuyten DS, Hastie T, Chi JT, Chang HY, van de Vijver MJ: Combiningbiological gene expression signatures in predicting outcome in breastcancer: An alternative to supervised classification. Eur J Cancer 2008,44(15):2319–2329.34. Chia SK, Bramwell VH, Tu D, Shepherd LE, Jiang S, Vickery T, Mardis E, LeungS, Ung K, Pritchard KI, Parker JS, Bernard PS, Perou CM, Ellis MJ, Nielsen TO:A 50-gene intrinsic subtype classifier for prognosis and prediction ofbenefit from adjuvant tamoxifen. Clin Cancer Res 2012, 18(16):4465–4472.35. Cheang MC, Voduc KD, Tu D, Jiang S, Leung S, Chia SK, Shepherd LE, LevineMN, Pritchard KI, Davies S, Stijleman IJ, Davis C, Ebbert MT, Parker JS, EllisMJ, Bernard PS, Perou CM, Nielsen TO: Responsiveness of intrinsicsubtypes to adjuvant anthracycline substitution in the NCIC.CTG MA.5randomized trial. Clin Cancer Res 2012, 18(8):2402–2412.36. Jorgensen C, Nielsen T, Bjerre K, Liu S, Wallden B, Balslev E, Nielsen D,Ejlertsen B: PAM50 breast cancer intrinsic subtypes and effect ofgemcitabine in advanced breast cancer patients. Acta Oncol 2013.37. Filipits M, Rudas M, Jakesz R, Dubsky P, Fitzal F, Singer CF, Dietze O, Greil R,Jelen A, Sevelda P, Freibauer C, Muller V, Janicke F, Schmidt M, Kolbl H,Rody A, Kaufmann M, Schroth W, Brauch H, Schwab M, Fritz P, Weber KE,Feder IS, Hennig G, Kronenwett R, Gehrmann M, Gnant M, EP Investigators:A new molecular predictor of distant recurrence in ER-positive, HER2-negative breast cancer adds independent information to conventionalclinical risk factors. Clin Cancer Res 2011, 17(18):6012–6020.38. Evaluation of Genomic Applications in Practice and Prevention (EGAPP)Working Group: Recommendations from the EGAPP Working Group: cantumor gene expression profiling improve outcomes in patients withbreast cancer? Genet Med 2009, 11(1):66–73.39. Kronenwett R, Bohmann K, Prinzler J, Sinn BV, Haufe F, Roth C, Averdick M,Ropers T, Windbergs C, Brase JC, Weber KE, Fisch K, Muller BM, Schmidt M,Filipits M, Dubsky P, Petry C, Dietel M, Denkert C: Decentral gene expressionanalysis: analytical validation of the Endopredict genomic multianalytebreast cancer prognosis test. BMC Cancer 2012, 12:456–2407. 12-456.40. Elloumi F, Hu Z, Li Y, Parker JS, Gulley ML, Amos KD, Troester MA:Systematic bias in genomic classification due to contaminating non-neoplastic tissue in breast tumor samples. BMC Med Genomics 2011,4(54):8794–4-54.41. Mee BC, Carroll P, Donatello S, Connolly E, Griffin M, Dunne B, Burke L,Flavin R, Rizkalla H, Ryan C, Hayes B, D’Adhemar C, Banville N, Faheem N,Muldoon C, Gaffney EF: Maintaining Breast Cancer Specimen Integrityand Individual or Simultaneous Extraction of Quality DNA, RNA, andProteins from Allprotect-Stabilized and Nonstabilized Tissue Samples.Biopreserv Biobank 2011, 9(4):389–398.42. Graham K, Ge X, De Las MA, Tripathi A, Rosenberg CL: Gene expressionprofiles of estrogen receptor-positive and estrogen receptor-negativebreast cancers are detectable in histologically normal breast epithelium.Clin Cancer Res 2011, 17(2):236–246.43. Clare S, Pardo I, Mathieson T, Lillemoe H, Blosser R, Choi M, Sauder C, Doxey D,Badve S, Storniolo A, Atale R, Radovich M: “Normal” tissue adjacent to breastcancer is not normal [abstract]. Cancer Res 2012, 72(24 Suppl):P1–03-02.doi:10.1186/1471-2407-14-177Cite this article as: Nielsen et al.: Analytical validation of the PAM50-based Prosigna Breast Cancer Prognostic Gene Signature Assay andnCounter Analysis System using formalin-fixed paraffin-embedded breasttumor specimens. BMC Cancer 2014 14:177.Submit your next manuscript to BioMed Centraland take full advantage of: • Convenient online submission• Thorough peer review• No space constraints or color figure charges• Immediate publication on acceptance• Inclusion in PubMed, CAS, Scopus and Google Scholar• Research which is freely available for redistributionSubmit your manuscript at www.biomedcentral.com/submitNielsen et al. BMC Cancer 2014, 14:177 Page 14 of 14http://www.biomedcentral.com/1471-2407/14/177


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items