UBC Faculty Research and Publications

Atlas – a data warehouse for integrative bioinformatics Shah, Sohrab P; Huang, Yong; Xu, Tao; Yuen, Macaire M; Ling, John; Ouellette, BF F Feb 21, 2005

Your browser doesn't seem to have a PDF viewer, please download the PDF to view this item.

Item Metadata


52383-12859_2004_Article_359.pdf [ 1.88MB ]
JSON: 52383-1.0224107.json
JSON-LD: 52383-1.0224107-ld.json
RDF/XML (Pretty): 52383-1.0224107-rdf.xml
RDF/JSON: 52383-1.0224107-rdf.json
Turtle: 52383-1.0224107-turtle.txt
N-Triples: 52383-1.0224107-rdf-ntriples.txt
Original Record: 52383-1.0224107-source.json
Full Text

Full Text

ralssBioMed CentBMC BioinformaticsOpen AcceDatabaseAtlas – a data warehouse for integrative bioinformaticsSohrab P Shah, Yong Huang, Tao Xu, Macaire MS Yuen, John Ling and BF Francis Ouellette*Address: UBC Bioinformatics Centre, University of British Columbia, Vancouver, BC, CanadaEmail: Sohrab P Shah - sohrab@bioinformatics.ubc.ca; Yong Huang - yongh@bioinformatics.ubc.ca; Tao Xu - taoxu@bioinformatics.ubc.ca; Macaire MS Yuen - mack@bioinformatics.ubc.ca; John Ling - jling@bioinformatics.ubc.ca; BF Francis Ouellette* - francis@bioinformatics.ubc.ca* Corresponding author    AbstractBackground: We present a biological data warehouse called Atlas that locally stores andintegrates biological sequences, molecular interactions, homology information, functionalannotations of genes, and biological ontologies. The goal of the system is to provide data, as wellas a software infrastructure for bioinformatics research and development.Description: The Atlas system is based on relational data models that we developed for each ofthe source data types. Data stored within these relational models are managed through StructuredQuery Language (SQL) calls that are implemented in a set of Application Programming Interfaces(APIs). The APIs include three languages: C++, Java, and Perl. The methods in these API librariesare used to construct a set of loader applications, which parse and load the source datasets intothe Atlas database, and a set of toolbox applications which facilitate data retrieval. Atlas stores andintegrates local instances of GenBank, RefSeq, UniProt, Human Protein Reference Database(HPRD), Biomolecular Interaction Network Database (BIND), Database of Interacting Proteins(DIP), Molecular Interactions Database (MINT), IntAct, NCBI Taxonomy, Gene Ontology (GO),Online Mendelian Inheritance in Man (OMIM), LocusLink, Entrez Gene and HomoloGene. Theretrieval APIs and toolbox applications are critical components that offer end-users flexible, easy,integrated access to this data. We present use cases that use Atlas to integrate these sources forgenome annotation, inference of molecular interactions across species, and gene-diseaseassociations.Conclusion: The Atlas biological data warehouse serves as data infrastructure for bioinformaticsresearch and development. It forms the backbone of the research activities in our laboratory andfacilitates the integration of disparate, heterogeneous biological sources of data enabling newscientific inferences. Atlas achieves integration of diverse data sets at two levels. First, Atlas storesdata of similar types using common data models, enforcing the relationships between data types.Second, integration is achieved through a combination of APIs, ontology, and tools. The Atlassoftware is freely available under the GNU General Public License at: http://bioinformatics.ubc.ca/atlas/Published: 21 February 2005BMC Bioinformatics 2005, 6:34 doi:10.1186/1471-2105-6-34Received: 04 September 2004Accepted: 21 February 2005This article is available from: http://www.biomedcentral.com/1471-2105/6/34© 2005 Shah et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.Page 1 of 16(page number not for citation purposes)BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34BackgroundOne important goal in bioinformatics is to integrate datafrom disparate sources of heterogeneous biological infor-mation. Data integration allows us to assemble targeteddata reagents for bioinformatics analyses, and to discoverscientific relationships between data. Most public reposi-tories of biological data focus on deriving and providingone particular type of data, be it biological sequences(e.g., GenBank [1], UniProt [2]), molecular interactions(The Biomolecular Interaction Network Database (BIND)[3-5], The Human Protein Reference Database (HPRD)[6]), or gene expression (The Stanford microarray data-base [7]). Integrating these disparate sources of data ena-bles researchers to discover new associations between thedata, or validate existing hypotheses.Several recent studies have demonstrated the power ofintegrative bioinformatics. Using data from genomicsequences and annotations, mRNA expression, and sub-cellular localization, Mootha et al were able to use bioin-formatics approaches to identify one of the disease genesresponsible for Leigh syndrome [8]. In another exampleof an integrative bioinformatics approach, Stuart et al usedexisting publicly available data to generate hypothesesabout the functional roles of gene sets [9]. These twoexamples illustrate the potential of querying integratedpublic data to reveal novel relationships.However, working with publicly available biological datacan be challenging due to the volume and complexity ofthe data types. With the proliferation of massive, publiclyavailable data sets, researchers need a way to readily accessthis data. Querying distributed data has inherent limita-tions such as the server resource restrictions of the remoteresource, concerns of secure data transmission over theinternet, and of course the actual logistics of querying dis-tributed resources. In such an environment, the distrib-uted search space is difficult to process in a high-throughput way, and requires complex queries to tietogether the heterogeneous data. Consequently, there is aneed for a data integration solution that facilitates searchand retrieval in an efficient, flexible, high-throughputmanner.Several active solutions are available that attempt to inte-grate data and that provide the tools to retrieve that data.We have grouped these existing systems into three majorcategories, based on how the data is stored and integrated:full record, SQL-based, and distributed.Full record systems like SRS [10] and Entrez [11] store theintact record in a table and extract specific fields to indexand cross-reference. SeqHound [12] is a powerful systemprogrammatically through application programminginterfaces APIs. Much like Entrez and SRS, fully intactrecords are stored in SeqHound, with specific fieldsindexed. The major advantages of SeqHound over Entrezis that it is locally installable and provides API access tothe data. SeqHound highlights the power and utility of alocally installable warehouse.SQL-based systems implement relational models to storedata. This allows SQL-level access to specific parts of thedata model, enabling detailed queries on the data forgreater specificity of results. The data in relational modelsare stored as primitive data types as opposed to storingfully intact records that need parsing or processing toaccess the parts therein. For example, sequences and theirannotated biological features can be stored in their ownfields in the database, permitting 'substring' operations toextract parts of the sequence that span a particular featuretype using SQL. Systems like EnsMart [13] and DBGET/LinkDB [14] provide data in a relational form, such thatthe power of SQL is at users' disposal. EnsMart's relationalback-end provides users with the ability to construct intri-cate queries on the data by taking advantage of SQL.Distributed systems implement software to access hetero-geneous databases that are dispersed over the internet.JXP4BIGI [15] have created a generalized method toaccess, extract, transform, and integrate distributed data.The tool acts as a middle-ware for constructing a localinstance of a data warehouse. This system is customizable,versatile and uses industry standard data modeling, distri-bution, and presentation software. BioMOBY [16] is asemantic-based system utilizing ontologies, and a servicesmodel to support user queries. TAMBIS [17], like Bio-MOBY, is also a semantic-based system, and is also serv-ice-model driven. These semantic web implementationsdo not house the data locally, but rather query the originaldata provider for available services before sending queriesto that particular data provider. These systems are quitepowerful for interrogating disparate data sources of infor-mation. However, a disadvantage is that large queries maytake a long time to return or may not be returned at all dueto server resource restrictions. As well, the level of dataintegration is only at the services level, rather than at afield-based level which can provide much better resolu-tion for queries.Atlas is a versatile, flexible, and extensible data warehousethat provides a solution to these challenges. Our approachestablishes common relational data models enabling thereuse of each class of data model to store all data of thesame type. For example, a single interaction data model isused to store information from any of the interaction dataPage 2 of 16(page number not for citation purposes)that stores Entrez information (fully annotated sequenceand structure information) locally and can be accessedsets such as BIND, MINT, EBI IntAct [18], Database ofInteracting Proteins (DIP) [19], and HPRD.BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34Instances of these data models, once populated by thesource data, can then be interrogated using the developedretrieval APIs. These APIs encapsulate the SQL calls usedfor fine granular access to the data. Furthermore, ontolog-ical information stored in these databases captures therelationships between the many data types. Finally, toolsare developed that capitalize on the API methods, to facil-itate application specific demands of end-users, rangingfrom simple queries of specific data types, to complexqueries that infer molecular interactions across species.Atlas then, is designed for use by a wide audience frombiologist to software developer.Construction and contentThe Atlas system is made up of five main parts: 1) thesource data, 2) the ontology system, 3) the relational datamodels, 4) the APIs, and 5) the applications (see Figure1). The following sections outline the Atlas architecture indetail.Source dataWe categorize the Atlas data sources into four maingroups: 'sequence', 'molecular interactions', 'gene relatedresources', and 'ontology' (Figure 1). Currently, the datasources that fall into these categories are: 'sequence', Gen-Bank, RefSeq [11], and UniProt ; 'molecular interactions',HPRD, BIND, DIP, IntAct, and MINT; 'gene relatedresources', Online Mendelian Inheritance in Man(OMIM) [20], LocusLink [11,21], Entrez Gene [22], andHomoloGene [11,23]; and 'ontology', NCBI Taxonomy[11,24], and Gene Ontology [25,26]. Table 1 lists each ofthe sources of data incorporated into Atlas, and providesURLs where those sources can be found. Note that Gen-Bank refers to the integrated records from the Interna-tional Nucleotide Sequence Database Collaboration(GenBank [11], DDBJ [27], and EMBL [28]).Relational data models (schema design)This section describes the composition of the data modelsof the source data included in Atlas. The data models wepresent here are implemented in MySQL [29], an opensource relational database management system (RDBMS).As such we only provide Data Definition Language (DDL)files that are compatible with MySQL. Currently there areno plans to port these to other RDBMS systems.OntologyOntologies serve to define the concepts and relationshipsboth within a system and between systems. This vocabu-lary of concepts and relationships is representative of agiven expert domain of discourse such as sequences, geneannotations, and taxonomy. In Atlas, ontologies are cate-gorized into two classes: Atlas defined ontologies andcally within Atlas, as well as to characterize concepts andrelationships implicitly defined by the GenBank SequenceFeature data model. External ontologies include suchthings as NCBI Taxonomy for organism classification,Gene Ontology for gene annotations enabling categoriza-tion of biological features based on function, process, andcellular component, and the Proteomics Standards Initia-tive Molecular Interaction Standard (PSI-MI) controlledvocabulary [30]. The Atlas internal ontologies contain def-initions of terms such as identifier types like accessionnumbers, GI numbers, PSI-MI terms and identifiers,PubMed identifiers, file format types like XML, relation-ship terms, and concepts like GenBank Sequence Featuresand Feature Qualifiers, Sequencing Techniques. This partof the Atlas ontology consists of three tables: Ontologywhich include terms and definitions, Ontology_type thatdefines ontology source and category, andOntology_Ontology which stores term-term relation-ships. Foreign key constraints are used to ensure dataintegrity. In contrast to these tightly integrated ontologies,two other external vocabularies are instantiated as inde-pendent MySQL databases: GO and NCBI Taxonomy.These ontologies, unlike the others, do not implementforeign key enforcements to the other database modules.As a result, when ontology terms are updated, referencesto deleted terms deemed to be invalid are kept in the sys-tem until such time a full data set reload is performed.The Atlas internal ontology exists largely to help describeSequence Features as they exists in the GenBank SequenceFeature model, as this is the primary data source for fea-tures. Neither the Open Biological Ontologies (OBO) [31]relationship terms, nor the Sequence Ontology (SO) [32]relationship terms suited our needs as a feature ontology.We utilize the basic relationships similarly found in OBOand SO, such as 'is-a', 'part-of', and 'inverse-of' but we alsodefine more specific terms such as 'is-synonym-of', 'refersto PubMed', 'feature-includes-qualifier', and 'gene-con-tains-promoter'. By defining these specific relationships,we simplify the ontology tree into a flatter structure that issimple to query. In addition, subject-predicate-object tri-ples are not explicitly defined in the internal ontology, butrather are assigned at loading-time as the GenBankSequence Feature data is parsed and stored into the data-base. The relationship terms are not necessarily complete,but sufficient for our needs, and as new relationships areencountered, these are added accordingly. For example,we mapped all 66 GenBank feature keys to an entry in ourOntology table, which has enabled us to do feature-levelqueries for any type of feature in GenBank, or genomes weannotated in-house. We caution the reader that it is gener-ally understood that not all GenBank features have thesame informational value, nor quality of information.Page 3 of 16(page number not for citation purposes)external ontologies. The Atlas defined ontologies are usedto represent the concepts and relationships found specifi-However, to capture the maximum amount of informa-tion, we chose to extract and store all annotated features.BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34Architecture of the Atlas data warehouseFigur  1Architecture of the Atlas data warehouse. The data integrated in Atlas are first downloaded as data files from the public repositories shown in the Data Source panel. These data files are then parsed and loaded into the MySQL relational data-bases using the Atlas loaders. The Atlas Databases panel shows the databases grouped by biological theme. These groups are sequences (green), molecular interactions (yellow); genes and functional categorization (blue); and ontologies (orange). For each database the available data retrieval methods are marked as SQL (S), C++ Atlas API (C), Java Atlas API (J), and Perl Atlas API (P). The Retrieval panel shows the flexible, layered architecture of the interfaces to the databases. Data can be accessed directly using the MySQL client with SQL statements, through the APIs in C++, Java, and Perl, and through the end-user appli-								   								    !"#$%% &' () *( +	,	    --) -  +   !.      ) -!./	/-+,	        Page 4 of 16(page number not for citation purposes)cations implemented in the Toolbox. The APIs can also be used to implement web-based tools, or standalone applications.BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34With the locations of the features stored in Atlas, sub-sequences of features can be extracted in a high-through-put manner using SQL, the APIs, or the toolbox applica-tions. This is particularly useful, for example, in extractingfeatures like non-coding RNAs from complete genomes,or regions spanning a particular gene of interest. We areactively integrating selected external ontologies, andexpanding our internal ontologies. Plans for ontologyintegration include the National Library of Medicine(NLM) MeSH term and the Microarray Gene ExpressionData (MGED) ontology [33]. We are evaluating theoption of adopting frame-based ontology representations,and existing ontologies such as TAMBIS Ontology (TaO)[17,34]. In the near future, we will release the Atlas ontol-ogy in other formats such as GO flat file, RDF/XML, andOWL. A complete list of the ontologies is available on theAtlas website, and we provide the MySQL dumps forthese: http://bioinformatics.ubc.ca/atlas/ontology/.Sequence modelThe schema for sequences is organized into three mainparts: sequence, which stores the sequence string and asso-ciated meta-data such as sequencing technique and mole-cule type; sequence identifiers, for which all identifiers thatappear in the records are stored (see Figure 2); and anno-tated sequence features, for which feature keys, qualifierkeys and values and feature locations are stored. Thoughoutput of features into General Feature Format Version 2information is stored in other tables and can be pulledtogether dynamically as a GFF2 record is being con-structed. For example, the BioID_type table contains thedatabase source information in its db_source field and theinternal Atlas Ontology table's term field which repre-sents the feature type used in the GFF2 output. However,to reflect the fact that features in such output are nowreconstructed from the Atlas system, we prefix the originalsource type with 'Atlas:', such as in 'Atlas:GenBank/Ref-Seq'. The reader will note that there are two differentOntology tables in Atlas. A more detailed explanation forthe motivation for having two different kinds of Ontologytables is described in the previous Ontology section. How-ever, in the context of sequence features, it is the internalAtlas Ontology table that is relevant.The sequence string is stored in the Sequence table. Addi-tional fields for: sequencing technique, tech, such asexpressed sequence tags (ESTs); molecule type, moltype,such as DNA, RNA, protein, and nucleic acid; sequencelength, length; the NCBI taxonomy identifier, taxonid;and the definition line, defline, are also stored in theSequence table. Fields such as taxonid, tech, and moltypecan be used separately, or in combination to produce cus-tomizable queries that return highly specific sets of data.Sequence identifiers, as with all other external identifiers,are managed through a layer of abstraction by associatingthem with internal identifiers within Atlas, which act asTable 1: Data sources included in Atlas.Atlas Data Source Summary Table *Data Source URL Data Format Atlas Update Update MechanismGenBank Sequence ftp://ftp.ncbi.nih.gov/ncbi-asn1/ ASN.1 Daily IncrementalGenBank Sequence ftp://ftp.ncbi.nih.gov/ncbi-asn1/ ASN.1 Release ReloadGenBank Refseq ftp://ftp.ncbi.nih.gov/refseq/ ASN.1 Daily IncrementalGenBank Refseq ftp://ftp.ncbi.nih.gov/refseq/ ASN.1 Release ReloadNCBI Taxonomy ftp://ftp.ncbi.nih.gov/pub/taxonomy/ Delimited Text Release ReloadHomoloGene ftp://ftp.ncbi.nih.gov/pub/HomoloGene/ Delimited Text Daily ReloadOMIM ftp://ftp.ncbi.nih.gov/repository/OMIM/ Delimited Text Daily ReloadGene ftp://ftp.ncbi.nih.gov/gene/ Delimited Text Daily ReloadLocusLink ftp://ftp.ncbi.nih.gov/refseq/LocusLink/ Delimited Text Daily ReloadUniProt ftp://ftp.uniprot.org/pub/databases/uniprot/knowledgebase/ XML Bi-weekly ReloadHPRD http://www.hprd.org/download/ XML Release ReloadMINT http://mint.bio.uniroma2.it/mint/ XML Release ReloadDIP http://dip.doe-mbi.ucla.edu/dip/Download.cgi XML Release ReloadBIND ftp://ftp.blueprint.org/pub/BIND/current/bindflatfiles/bindindex/ Delimited Text Release ReloadGO http://www.godatabase.org/dev/database/archive/latest/ MySQL dump Release Reload* up-to-date information about data sources and statistics are available from the Atlas website http://bioinformatics.ubc.ca/atlas/Page 5 of 16(page number not for citation purposes)(GFF2) [35] is supported, the Feature table, one will note,does not explicitly contain source or type fields. Thisprimary keys. Having a single internal identifier for asequence allows us to relate all other identifiers found inBMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34the record to each other. In addition, ontologies for alltypes of identifiers currently found in GenBank ASN.1data files, as well as relationships between theseidentifiers are modeled in the Bioid and Bioid_Bioidtables, respectively. As mentioned above, sequence fea-tures are also modeled in Atlas. For details please refer tothe Ontology section below.Molecular interactionsFor molecular interaction data, we developed a relationalmodel compliant with the PSI-MI. Adopting a commonferent sources, and allows us to develop a set of commoninteraction retrieval APIs.Currently, HPRD, BIND, DIP, IntAct and MINT areincluded as interaction data sources. BIND, DIP, MINTand IntAct release their data in PSI-MI format. HPRD isreleasing data in both PSI-MI standard format, and theirown XML format. At the time of this publication, BINDreleased data as indexed flat files, ASN.1, XML, and PSI-MIformat (level 2).Atlas database schemaFigure 2Atlas database schema. There are four major functional groups. Biological Sequences: includes instances of GenBank sequences, RefSeq sequences, and UniProt sequences; Molecular Interactions: includes instances of BIND, HPRD, DIP, IntAct and MINT; Gene Related Resources: includes instances of OMIM, Entrez Gene, and LocusLink, and HomoloGene; and Ontol-ogy: includes instances of Taxonomy, Atlas internal ontologies, Gene Ontology, and PSI-MI ontologies.Gene_synonymPK gene_synonym_idFK1 geneidsynonymGene2unigenePK gene2unigene_idFK1 geneidunigene_clustergene2mimPK gene2mim_idFK1 geneidmimidGene_dbxrefPK gene_dbxref_idFK1 geneiddatabase_sourceid_valueGene_historyPK gene_history_idgeneiddiscontinued_geneiddiscontinued_symbolGene2pmidPK gene2pmid_idFK2 geneidpmidGene2accessionPK gene2accession_idFK1 geneidstatusrna_accessionrna_giprotein_accessionprotein_gigenomic_accessiongenomic_gigenomic_startgenomic_endorientationGenePK geneidtaxonidsymbollocustagchromosomemap_locationdescriptiongene_typegenerif_pmidPK generif_pmid_idFK1 gnenrif_idpmidEntrezGeneGene Related ResourcesTissuePK tissue_idFK1 dbxref_idtissuexref_valueInteractorPK interactor_idFK1 organism_idshortlabelfullnamesequencemoltypeExperiment_DbxrefPK,FK1,I1 experiment_idPK,FK2 dbxref_idPK erxperiment_dbxref_idvalueInteractor_CelltypePK,FK1,I1 celltype_idPK,FK2,I3,I2 interactor_idExperiment_DetectionPK,FK2,I1 experiment_idPK,FK1 detection_idPK experiment_detection_iddetection_typeInteractor_CompartmentPK,FK1,I1 compartment_idPK,FK2 interactor_idInteractionPK interaction_idFK1 interaction_type_idshortlabelfullnameconfidence_valueconfidence_unitdescriptionInteractor_DbxrefPK interactor_dbxref_idFK1,I5,I3 dbxref_idFK2 interactor_idvalueInteractor_InteractionPK interactor_interaction_idFK1,I1 interaction_idFK2,I3,I2 interactor_idroleistaggedisoverexpressedconfidence_valueconfidence_unitCelltypePK celltype_idFK1 dbxref_idcelltypexref_valueInteraction_DbxrefPK,FK1 interaction_idPK interaction_dbxref_idPK,FK2 dbxref_idvalueCompartmentPK compartment_idFK1 dbxref_idcompartmentxref_valueInteractor_TissuePK,FK1,I1 tissue_idPK,FK2,I3,I2 interactor_idDbxrefPK dbxref_idid_typedb_sourceDetectionPK detection_idshortlabelfullnamedescriptionInteraction_typePK interaction_type_iddescriptioninteraction_typeExperimentPK experiment_idshortlabelfullnameconfidence_valueconfidence_unitdescriptionDetection_DbxrefPK,FK1 detection_idPK,FK2 dbxref_idPK detection_dbxref_idvalueInteraction_ExperimentPK interaction_experiment_idPK,FK1 interaction_idPK,FK2 experiment_idFeaturePK feature_idshortlabelfullnameFK1 interactor_interaction_idLocationPK location_idFK1 feature_idbeginbeginintervalendsendintervalpositionpostionintervalsiteFeature_DbxrefPK feature_dbxref_idFK1 dbxref_idFK2 feature_idvalueOrganismPK organism_idshortlabelfullnametaxonidExperiment_OrganismPK experiment_organism_idFK2 experiment_idFK1 organism_idExperiment_Organism_CompartmentPK experiment_organism_compartment_idFK1 compartment_idFK2 experiment_organism_idExperiment_Organism_CelltypePK experiment_organism_celltype_idFK1 celltype_idFK2 experiment_organism_idExperiment_Organism_TissuePK experiment_organism_tissue_idFK1 tissue_idFK2 experiment_organism_idOntologyTaxon_namePK taxon_name_idFK1 taxon_node_idtaxonidnameunique_namename_classTaxon_nodePK taxon_node_idtaxonidparent_taxidrankembl_codeinherited_div_flagmit_genetic_code_idinherited_MGC_flagGB_hidden_flaghidden_subtree_flagcommentsFK1 genetic_code_idFK2 division_idTaxonomyGenetic_codePK genetic_code_idabbreviationnamecdestartDivisionPK division_iddivision_cdedivision_namecommentsHomologeneHomologenePK homologene_idHIDtaxonidgene_idgene_symbolprotein_giprotein_accessionAttributePK attribute_idnamevalueFK1 experiment_idFeature_DetectionPK feature_detection_dPK,FK1 feature_idPK,FK2 detection_idBiological SequencesBioidPK bioid_idid_stringFK1 bioid_type_idBioID_typePK bioid_type_idtypedescriptiondb_sourceBioid_SequencePK bioid_sequence_idFK1 bioid_idFK2 sequence_idSequencePK sequence_idsequencedeflinemoltypelengthdivisiontechtopologyreprversiontaxonidtimestampFeaturePK feature_idFK1 ontology_idFK2 bioid_idOntologyPK ontology_idFK1 ontology_type_idtermdescriptionatlas_idexternal_idBioid_BioidPK bioid_bioid_idFK2 bioid_id1FK1 bioid_id2FK3 ontology_idLocationPK location_idFK1 feature_idstartstopstrandrankFK2 ontology_idQualifierPK qualifier_idFK1 feature_idvalueFK2 ontology_idOntology_OntologyPK ontology_ontology_idFK1 ontology_id1FK2 ontology_id2descriptionBlobfilePK blobfile_idFK1 bioid_idblob_fileontology_idtimestampFK2 load_stat_id Ontology_typePK idbterm_type_idontology_typedescriptionLoad_statPK load_stat_idtotal_secondsfilenamerecordstimestampCommentPK comment_idFK1 bioid_idtextrankOntologyFeature_FeaturePK feature_feature_idFK1,I1 feature_id1FK2,I2 feature_id2FK3,I3 ontology_idSequenceMolecular InteractionsOntologyPK ontology_idFK1 ontology_type_idtermdescriptionatlas_idexternal_idOntology_OntologyPK ontology_ontology_idFK1 ontology_id1FK2 ontology_id2descriptionFK3 ontology_idOntology_typePK ontology_type_idontology_typedescriptionGenerifPK gnenrif_idFK1 geneidlast_updatedescriptiontermPK idnameterm_typeaccis_obsoleteis_rootterm2termPK idFK1 relationship_type_idFK2 term1_idFK3 term2_idterm_definitionPK,FK1 term_idterm_definitionFK2 dbxref_idterm_commentreferenceterm_dbxrefPK,FK1 term_idFK2 dbxref_idis_for_definitiondbxrefPK idxref_keyxref_keytypexref_dbnamexref_descterm_synonymPK,FK1 term_idterm_synonymacc_synonymFK2 synonym_type_idgraph_pathPK idFK1 term1_idFK2 term2_iddistanceGene OntologyOmimPK omim_idmonth_enteredday_enteredyear_enteredlocationspace1commentsspace2OmimPage 6 of 16(page number not for citation purposes)interaction data model allowed us to unify data from dif- The Atlas interaction model consists of four major enti-ties: Interactor, Interaction, Experiments and Dbxref.BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34Interactor holds information about one of the interactingmembers in an interaction, such as the interactor's name,taxonomy, sequence, molecular type, features, subcellularlocalizations, and external identifiers. An Interaction con-sists of one or more interactors, and one or moreexperiments.Experiment stores information about the experimentsused to identify interactions. Finally, Dbxref is used tocrosslink the external identifiers such PubMed id, RefSeqaccession, HPRD id, BIND id, and Ontology id, for exam-ple (see Figure 2). As an additional note, the Feature tablein the Interaction database is mainly used to store proteinfeatures involved in the interactions.We will release a PSI-MI level 2 compliant version of theInteraction model, and API upon public release of thelevel 2 specification.Gene related resourcesWe integrated OMIM, LocusLink, Entrez Gene, Homolo-Gene and the annotation part of GO into the Atlas systemin order to provide gene-related information. Conven-iently, the OMIM and LocusLink data sources provide flatfile tables which could be imported directly with theMySQL import function. Entrez Gene will eventuallyreplace LocusLink, however in order to maintain asmooth transition and backward compatibility, we aremaintaining populated relational models for both EntrezGene and LocusLink until LocusLink is officially retired.Integration between HomoloGene and Sequence isachieved by relating the taxonomic, protein sequence andgene identifiers with Atlas' Bioid table. This allows us tointegrate these databases and provide linkage between, forexample, orthologous genes present in different interac-tion scenarios (see Utility of the Atlas system).Application programming interfacesThere are two classes of APIs in Atlas: loader and retrieval.Components of Atlas for which we have developed ourown relational models, such as the Biological Sequencescomponent, or the Molecular Interactions component,each have their own set of loader APIs. The loader APIsused to build the loading applications, populate instancesof the relational models in the Atlas databases. Thoughmost end-users will never need to use the loader APIs,they are critical to the implementation of the Atlas loadingprocess, and are provided to the software developmentcommunity. The other class of APIs are the retrieval APIs.These APIs serve to retrieve the data stored in Atlas. Theyare required for developing custom retrieval applicationssuch as the Atlas toolbox applications. The loader API forBiological Sequences is implemented in C++ as it relieshand, is provided in all three languages: C++, Java, andPerl. The Java and Perl APIs return sequences as BioJavaSimpleSequence and BioPerl Bio::Seq objects, respec-tively. The loader and retrieval APIs for Molecular Interac-tions are provided in Java. Though retrieval APIs are notsupported in all languages, further development in Perland C++ will be added if our user community requeststhem. Please refer to Figure 1 for a mapping of data mod-ules to currently supported programming languages. Theproject is also open source and other developers areencouraged to contribute. All the transactions betweenthe APIs and the database are specified by the numerousSQL statements which are all defined within the majorityof the API methods.Application programming interface architectureThe API is constructed using object-oriented methodolo-gies, employing objects to represent everything from low-level database connections to high-level data structures,and their access methods. This is illustrated in Figure 3.Common in the design of the C++, Java, and Perl APIs, area set of APIs written for MySQL database connectivitywhich handles the opening and closing of MySQL connec-tions, as well as managing the execution of the SQL state-ments themselves. All subsequent APIs that interact withthe Atlas database extend from this set of APIs.Both the data loader and the retrieval utilities share a com-mon class responsible for low-level data transformations.This class includes methods that facilitate conversionsbetween two internal Atlas identifiers, such as bioid_id toontology_id, or methods that convert internal Atlasidentifiers to externally referenced public identifiers, suchas GenBank accession numbers, or GI numbers. Inherit-ing this shared identifier conversion class benefits boththe loader APIs and the retrieval APIs, by providing themwith the necessary tools to integrate information.The Biological Sequences component of Atlas managescommon identifiers, and hash maps in the Seq class. Thisclass is inherited by both the SeqLoad class and SeqGetclass, which define the loader methods, and retrievalmethods, respectively. Another feature of the BiologicalSequences API, is its ability to control stream output basedon molecule types. API users simply specify which mole-cule type to filter by, through calls to higher-level retrievalmethods, and SeqGet will then handle the logistics ofstream management. Similarly with the Molecular Inter-actions component of Atlas, the InteractionDb class isinherited by the InteractionLoad class and the Interac-tionGet class, respectively defining the loader andretrieval methods which manipulate the data in memory.Page 7 of 16(page number not for citation purposes)heavily on the NCBI C++ Toolkit [36] to parse the ASN.1data. The Biological Sequence retrieval API, on the otherBMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34Atlas API architectureFigure 3Atlas API architecture. MySqlDb, Seq, SeqGet and SeqLoad classes/modules (grey) are available in all three languages: C++, Java, and Perl. The SeqLoad and Seqloader modules are created in C++ only as these are tightly coupled to the NCBI C++ Toolkit. All other classes are available in Java. Applications share the common modules SeqLoad, SeqGet, InteractionLoad, and InteractionGet which provide the methods necessary for loading and retrieval operations, to and from the databases. These modules employ additional classes (not shown) that are representative of the major data model components such as Sequence, Interaction, Interactor, and Dbxref, for example.		 			  				 	!"#		!	#				Page 8 of 16(page number not for citation purposes)BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34Our Java interaction APIs, for example, are tightly coupledto our interaction data model with classes representing allthe major schema objects such as Interaction, Feature,Dbxref, and Experiment. The APIs are works in progressand we continue to develop and improve them. We areconsidering even more tightly coupled API developmentby using XML schema code generators such as JAXB.All the source code is provided under the GNU Generalmodel future API development on numerous functionswe have already implemented.ApplicationsToolboxThe Atlas toolbox is a collection of applications that usethe C++ API to perform common sequence and featureretrieval tasks. The applications are standard Unix com-mand-line based tools that follow a command-lineTable 2: Atlas toolbox applications.Application Function Input OutputSequenceac2seq Retrieve sequences given an accession Nucleic acid or Protein Accession Number(s) Sequences in Fasta formatfeat2seq Retrieve sub-sequences that span features Feature type (and qualifier) Sequences in Fasta formatgi2seq Retrieve sequences given a GenInfo identifier GenInfo Identifier(s) (GI Number(s)) Sequences in Fasta formatgi2seqentry Retrieve sequences given a GenInfo identifier GenInfo Identifier(s) (GI Number(s)) GBFF, EMBL, GFF, FTABLE, ASN.1, GBSEQtax2seq Retrieve sequences by taxonomy NCBI taxon identifier or scientific name of taxonSequences in Fasta formattech2seq Retrieve sequences by sequencing technique Sequencing technique (eg EST, GSS, etc.) Sequences in Fasta formattechtax2seq Retrieve sequences by taxonomy and sequencing techniqueSequencing technique and NCBI taxonid/scientific name of taxonSequences in Fasta formatLoaderfastaloader Fasta sequence data loader Sequences in Fasta formatseqloader ASN.1 sequence data loader GenBank/RefSeq ASN.1 recordsFeatureac2feat Retrieve features GenBank Accession number (s) Features in GFF or FTABLE formatgi2feat Retrieve features GenInfo Identifier(s) (GI Number(s)) Features in GFF or FTABLE formatTaxonomyac2tax Retrieve taxonomy given an accession numberGenBank Accession number (string) NCBI taxon identifier (integer)gi2tax Retrieve taxonomy given a GenInfo identifier GenInfo Identifier (integer) NCBI taxon identifier (integer)ID Convertersac2gi Convert an accession number to a GenInfo identifierGenBank Accession number (string) GenInfo Identifier (integer)gi2ac Convert a GenInfo identifier to an accession numberGenInfo Identifier (integer) Accession number (string)tax2gi Retrieve GenInfo identifiers associated with taxon identifierNCBI taxon identifier (integer) GenInfo Identifier (integer)Page 9 of 16(page number not for citation purposes)Public License (GPL), and therefore any developer can option-based interface for parameter entry. These are end-user applications and do not require any programmingBMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34ability to use them. We have developed toolbox applica-tions for sequence retrieval from accession and GI num-bers, retrieval of sequences from all organisms beneath agiven node in the NCBI taxonomy tree, retrieval of fea-tures given accession and GI numbers, retrieval of sub-sequences corresponding to specific features identified byqualifiers and their values, and retrieval of a set of interac-tions associated with a molecule given the accessionnumber of an Interactor. Besides being useful tools, thetoolbox applications' source code provides good exam-ples of application development using the APIs. Softwaredevelopers wishing to use the APIs can use these toolboxapplications as a starting point for their own customapplications (see Table 2).Data loadersData loaders are provided in Atlas to facilitate the parsingand loading of the source datasets into their respectiveAtlas database tables. Two main classes of loaders are cur-rently supplied in the Atlas package: sequence loaders andinteraction loaders. Though other types of data are loadedinto Atlas, their loading is trivial as MySQL databasedumps of these datasets are already provided by the dataproviders.The first class of loaders is the sequence-based loaders.Within this class there are two applications provided:seqloader and fastaloader. The seqloader performs themajority of the sequence loading from GenBank andRefSeq datasets. These datasets have long been repre-sented as ASN.1 (binary/text) by the NCBI [37], and arecompact and well defined for storing of structured data.The seqloader was built using the NCBI C++ SoftwareDevelopment Toolkit [36] which was designed to specifi-cally parse the ASN.1 sequence data, extracting suchthings as the sequence, associated identifiers, features ofthe sequence and related publications. There are, how-ever, instances where sequence data is missing from theASN.1 records. In these situations, we obtain the missingrecords from the NCBI Entrez system in the form of Fastarecords. The fastaloader application is then used to updatethe sequence field in Atlas with the sequences from theFasta records.The second class of loaders is interaction-based loaders.These loaders are exclusively implemented in Java. Thedatasets loaded by this class of loaders include BIND,HPRD, MINT, IntAct and DIP. All the interaction loadersare designed to parse the data in the way that best dealswith that particular source data's structure and content(mostly XML). The interaction data is loaded using a com-mon interaction object model, and the interaction load-ing APIs provide a flexible and extensible framework forBesides these classes of loaders, there is also a Java basedloader that parses and loads UniProt sequence data. Inaddition, scripts are used to load datasets for whichMySQL dumps, or tab-delimited database dumps are pro-vided. This is handled using the MySQL import function,and eliminates the need to devise special parsers andloaders.GenBank and RefSeq are checked daily for incrementalupdates from the NCBI. Accession numbers are used tomaintain the integrity of the data. New accession numbersreflect new records and will be inserted into the database.Updated sequences or records with same root accessionnumber and patched annotations will replace existingrecords in the database. When new releases of GenBank/RefSeq are made available, all databases are purged andreloaded to remove retired records and to maintain refer-ential integrity.Web toolsThough we encourage the use of Atlas as an in-houserepository, it can also act to serve the wider internet com-munity. We provide a publicly available web interface tothe Atlas databases to demonstrate some of its functional-ity. This offers basic access to GenBank, RefSeq, NCBI Tax-onomy, Atlas Ontologies, BIND, HPRD, MINT, IntAct andDIP. Web interfaces to the Atlas toolbox applications:ac2gi, ac2seq, ac2tax, feat2seq, gi2ac, gi2feat, gi2seq,gi2tax, tax2seq, techtax2seq, tech2seq are available. Inaddition, interacting partners for proteins identified byaccession numbers or GI numbers can be retrieved fromany of the four interaction databases stored in Atlas. Theseweb tools can be found at: http://bioinformatics.ubc.ca/atlas/webtools/.Utility of the Atlas systemThe Atlas data warehouse offers maximum flexibility ofdata retrieval and integration. Users can access data inAtlas at the SQL, API and end-user application levels. Rou-tine, pre-defined queries can be accessed through the APIsin Java, C++, and PERL (see API section, above), enablingdevelopers to incorporate these queries in their softwareapplications. Most of these queries have been used tobuild the Atlas toolbox, a set of end-user applications thatrun on the Unix command-line (Table 2). Included in thetoolbox are common utilities for converting GenBankASN.1 sequences to file formats supported by the NCBIToolkit [1] such as XML, GenBank Flat File, and FASTA. Inaddition, information regarding features that are anno-tated on sequence records can be exported as General Fea-ture Format Version 2 (GFF2). The recently developedGeneral Feature Format Version 3 (GFF3) is not currentlysupported in Atlas, to allow its specification time to stabi-Page 10 of 16(page number not for citation purposes)future interaction data loading efforts. Currently, we aredeveloping a PSI-MI level 2 data loader.lize. However, its support in Atlas is planned in futurereleases. In the following sections, we illustrate use-casesBMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34of the system at the SQL, API and toolbox levels with spe-cific biological themes in mind.Single record queriesSingle record queries are the simplest use case of the sys-tem. Users can input a GenBank or RefSeq accessionnumber and/or GI number into the ac2seq and gi2seqtoolbox applications to retrieve the relevant sequencerecord in Fasta, GenBank or ASN.1 format. Features on aparticular sequence can also be retrieved independentlywith GenBank or RefSeq accession numbers and/or GInumbers. The single record queries can also be performedin batch mode where the user supplies a list of accessionnumbers or GI numbers and all data pertinent to the listof identifiers is then retrieved.Genome annotationAtlas provides tools to generate data reagents for genomeanalysis as well as a data model for storing biological fea-tures that have been annotated on the sequences. Coupledwith Pegasys [38] and Apollo [39], the Atlas system is anessential part of our annotation platform (see Figure 4).Atlas functions simultaneously as a data reagent generatorfor sequence alignment analysis, a storage system forannotations that are to be submitted, and a data transfor-mation tool that can convert Apollo-compatible data toNCBI submission tool compatible data.Atlas provides users with the ability to generate customsets of data to use as reagents. For example, using tax2seq,users can input a specific node of the NCBI taxonomy treeusing its scientific name, or its NCBI taxonomy id andretrieve all nucleotide and amino acid sequences fromorganisms in the tree rooted at that node. This has specialutility in genome analysis where specific sets of data fromclose relatives of the genome of interest enable compara-tive genomic methods for functional annotation. Further-more, this type of taxonomy querying can be combinedwith the 'tech' field in the NCBI data model to producesequences derived from different sequencing techniquessuch as expressed sequence tags (EST), genome surveysequence (GSS), sequence tagged sites (STS), highthroughput genomic (HTG), etc. Compiling these specificdata sets allows the user to perform more directedsequence similarity searches, for example, that yield morespecific hits.Using the sequence data structure to model existing anno-tations in sequence records, Atlas can be used to storeadditional annotations created in Sequin [40] and Apollo[39]. We have built a GAME XML [41] loader that storesannotations exported from Apollo. When used for thispurpose, Atlas serves as a holding bay for sequences thatmulti-level query system provided by the Atlas APIs (seeFigure 4). Additionally, the annotations stored from aGAME XML [41] file are exportable in GFF2, or SequinFeature Table Format [42] for use with NCBI submissiontools like tbl2asn [42].Inference of protein-protein interactionsDeriving new associations from the information extractedfrom Atlas has proven to be particularly useful in develop-ing a prototype system that infers interactions across spe-cies, detailed in "Ulysses – an Application for theProjection of Molecular Interactions across Species"(Kemmer D: in preparation, from the Wasserman andOuellette laboratories).Given that the data for protein-protein interactions foundwithin model organisms can be extremely sparse, Ulyssesemploys homology information to help bridge the gaps inthe interaction data by projecting known interactions inone species onto other species for which those interac-tions are not known, and subsequently inferring poten-tially novel interactions in those species. Ulysses is able toperform its analyses and inferences, in part, bycapitalizing on the integration, offered by Atlas, of HPRD,BIND, and HomoloGene. Atlas makes it possible toretrieve interactions for one species known to occur inanother species, by integrating these datasets under onequery space, and by providing the API and tools whichmake such queries simple.As an example, in both the MINT and DIP databases, pro-tein C-C chemokine receptor type 3 (SwissProt accessionnumber P51677) was found to interact with protein Smallinducible cytokine A24 precursor (SwissProt accessionnumber O00175) in human (MINT interaction 14962;DIP interaction 10472E). Although referenced bydifferent publications ([43], [44]), both interactions arelikely to be the same. With corroborating evidence forthese seemingly synonymous interactions, it can beclaimed with more certainty that two proteins do indeedinteract. Furthermore, homologs for both sequences canbe found in mouse and rat through HomoloGene.Though these homologs are not found to be interactingpartners in either mouse or rat, it is reasonable to specu-late that such interactions exist in both these organisms.Disease-gene associationsThe Atlas system is also being used to determine yeastorthologs of genes that are implicated in human disease(Hieter P: in preparation). The inference being thathuman genes for which there are yeast orthologs representessential genes which are candidates for human diseaseagents. Compiling the reagents for this custom databasePage 11 of 16(page number not for citation purposes)can be submitted to DDBJ, EMBL, or GenBank in a rela-tional form that can be mined in the interim using thewas straightforward using the Atlas tools. It takes advan-BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34tage of the linkage between sequence identifiers, Taxon-omy, HomoloGene, and OMIM.Discussionaccess to the data by means of SQL queries, API-level que-ries, and end-user application-level queries. Our goal wasto create a system that serves as a platform through whichinformation from many sources of data can be interro-Using Atlas in genome annotationFigure 4Using Atlas in genome annotation. Atlas facilitates genome annotation at multiple levels: creation of data reagents, stor-age of annotations, and data transformation for submission. Here we show a schema of our genome annotation process that integrates Pegasys, Apollo, NCBI tools and Atlas into a comprehensive platform. Data reagents for sequence alignment are compiled using the Atlas toolbox applications. Computational analyses are run through the Pegasys system which outputs GAME XML for import into Apollo. Annotations are saved in a GAME XML which are then imported into Atlas using the GameLoader. At this step, the biological features created in the annotation process are stored in the Atlas Feature tables, exactly the same way a GenBank sequence record containing annotations are stored. These annotations can then be retrieved using the Atlas toolbox application ac2feat and exported in GFF2 or Sequin Feature Table Format for import into the NCBI submission tools for validation, and submission to GenBank.														 	Page 12 of 16(page number not for citation purposes)We have built a data warehouse of biological informationwith the goal of providing high-throughput, flexiblegated, enabling biologists and computer scientists to eas-ily carry out queries necessary for their research. The dataBMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34warehouse facilitates complex queries on local instancesof GenBank, RefSeq, UniProt, HPRD, BIND, NCBI Taxon-omy, HomoloGene, Gene Ontology, OMIM, Entrez Gene,and LocusLink. With previously disparate data nowunified in a relational model, SQL can be used to retrievethis consolidated information at once. Though Atlas canact to serve data publicly over the internet, its simple setupenables anyone or any institution to easily serve their owncustomized data warehouses to their own local users.Installing Atlas in-house to serve local users gives the dataprovider full control over the data they serve. Giving usersaccess to the system on a high-bandwidth internal net-work offers convenience and high-performance for largequeries, such as retrieving all human ESTs. Such data isthen more readily retrieved with lower latency and higherbandwidth than attempting to retrieve the same data overthe internet.One of the important strengths of the Atlas architecture isthat it allows data integration at two levels. The first leveluses a common data model to integrate similar types ofdata from different sources (e.g., GenBank or UniProt,and BIND or HPRD). The second level uses the APIs,ontologies, and tools to cross-reference disparate types ofdata.For example, consider the task of retrieving all amino acidsequences, and from all organisms found within the taxo-nomic tree rooted at a given taxonomic node (e.g., verte-brata), from the RefSeq database. With a single call to thetaxonName2Sequences method, the user can accomplishthis task. Within these API methods are SQL statementswhich first retrieve the taxonid from the Taxonomy data-base. Then using a recursive method, the taxon identifiersfor all organisms beneath that given taxon node, arereturned. All amino acid sequences for each of these taxonidentifiers are then retrieved using taxonId2Sequences(see API documentation [45] for more details).Uniting disparate sources of data is a useful exercise thathighlights the challenges that the data itself presents. Anychanges to the source data structure often requires soft-ware code changes in order to properly parse the new dataformat. Failure to do so often leads to the inability to loadat least some of the information, if not all. Furthermore,the quality of the original data may often be imperfect asmuch of this data is curated manually, and hence is sub-ject to data entry errors. Everything from missing data toimproperly spelled key terms can impede the loadingprocess. For this reason, it is essential to devise a systemthat is robust enough to handle unforeseeable exceptions.Policies on how to handle such exceptions are importantto define and implement. We try to adhere to the carefulers for remediation. This is especially important when thespecifications of the data are already strictly defined, yetare not followed, or are being misinterpreted.Semantic inconsistencies may arise due to differences inthe interpretation of biological concepts and data, anddifferences in how such information is mapped into anintegration system. That is to say, two systems may con-tain different data for the same semantic entity. For exam-ple, two interaction databases containing localizationdata for the proteins stored within, may indicate conflict-ing localization information for a given protein if the setof experimental evidences, used to determine localiza-tion, were different between the two systems. Such con-flicts between data source providers pose challengesduring the integration process as decisions need to bemade to resolve the conflict. We continue to evaluatemethods of resolving such conflicts. One simple solutionis to store the information from all sources as is, and alsoannotate that information with the source from which itcame, so as not to have any information loss. In this way,users can decide on which source they believe and poll thedata accordingly. Another solution, which is not as clearcut, would be to selectively merge data, pruning thosefacts we determine to be incorrect (perhaps based onsome measure of consensus between multiple systems),thus leaving only one instance of a factoid in our data-base. However, as it would not necessarily be our goal tojudge the correctness of data, this is perhaps a task betterleft to users of our system.Comparison with other systemsSeveral other systems are available which have similargoals and provide good solutions to the problem of dataintegration. We have chosen to discuss Atlas in the contextof three other systems: Entrez [11], SeqHound [12] andEnsMart [13]. The Entrez system, produced by the NCBI,provides "an integrated database retrieval system that ena-bles text searching, using simple Boolean queries, of adiverse set of 20 databases". This web-based system isextremely extensive in the scope of data it provides, and infact many of the Atlas data sources originate from NCBI(GenBank, RefSeq, HomoloGene, Taxonomy, OMIM,Entrez Gene, and LocusLink). The Entrez resources can befound on the NCBI website [46]. In contrast to Entrez,Atlas warehouses the data locally, obviating the need forlow-throughput, internet-based queries. Also, additionaldata sets like HPRD, DIP, MINT and BIND, not currentlyavailable through the Entrez interface, have been added toAtlas.SeqHound [12] is a database of biological sequences andstructures, developed by the Blueprint Initiative [47].Page 13 of 16(page number not for citation purposes)logging of incorrect entries that we find during the loadingprocess, and to promptly report these to the data provid-SeqHound also stores information on OMIM, LocusLink,and Gene Ontology. SeqHound and Atlas warehouse sim-BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34ilar data types. SeqHound provides some different datathan Atlas (most notably MMDB). For interaction data,SeqHound utilizes the BIND database. In contrast, Atlasstores interaction data from a number of sourcesincluding BIND, HPRD, MINT, DIP, and IntAct. Atlasthen, is a more comprehensive repository of interactiondata. The major difference between SeqHound and Atlasis in their architectural design. SeqHound stores fullrecords and indexes specific fields which are extractedupon loading. In contrast, Atlas provides relational mod-els for all data sources. This allows SQL-level access to spe-cific parts of the data model. The data in the Atlasrelational models are stored as primitive data types asopposed to storing whole records that need parsing orprocessing. For example, sequences and their annotatedbiological features can be stored in their own fields in thedatabase, permitting 'substring' operations to extract partsof the sequence that span a particular feature type usingSQL. Other systems like EnsMart [13] and the UCSCgenome browser [48] have also adopted fully relationalmodels. These systems also provide SQL access over thefull data model, and allow arbitrarily complex queriessimilar to Atlas.EnsMart is a software system designed by EMBL-EBI [49]and the Sanger Institute [50] which produces and man-ages automated annotations. The focus of EnsMart isslightly different than Atlas in that its 'core' data is fullysequenced eukaryotic genomes. While information onthese genomes is extremely rich in EnsMart and well-inte-grated using relational models, Atlas attempts to providea much more extensive source of sequence information.This enables researchers interested in bacteria, viruses,plants or humans to access the system and sources of inte-grated data with equal facility.The Atlas system is designed to be locally installed and isnot a data provider per se, but rather an engine that shouldbe accessed 'in-house'. As with any locally-installable sys-tem of this nature, significant time and hardwareresources are needed to make the system functional. Theutility of the Atlas system will far outweigh the setup timerequired to get it up and running. Currently, API access toAtlas is limited to the users at the UBC BioinformaticsCentre, University of British Columbia, however the webtools are available worldwide.Future workWhen working with sources of data from different dataproviders (for example UniProt and RefSeq), it is advanta-geous to create mappings from one data source to theother to prevent redundancy and to make associationsbetween proteins to map annotations from one source todifferent sources that are referring to the same proteinproduct to a single identifier.We will constantly monitor and adjust any change of thedata sources. In the near future, we will provide supportfor a PSI-MI level 2 release, and complete the migration ofLocusLink to Entrez Gene. In addition, we are expandingAtlas to include other sources of data. We are currentlyadding MEDLINE, dbSNP and pathway data to support anintegrative genomics and clinical informatics initiative,currently underway in our laboratory. With Atlas in hand,we are also working on an integration project that super-imposes co-expression networks derived from microarrayexperiments and protein-protein interaction networks, toestimate the utility of co-expression networks in inferringprotein interactions.ConclusionAtlas is a data warehouse that enables high-throughput,flexible and complex queries on biological data. The sys-tem integrates sequences, molecular interactions, taxon-omy and homology, and functional annotations ongenes. The system functions as data infrastructure to sup-port bioinformatics research and development. Atlas iscurrently being used in genome annotation projects, dis-ease-gene association projects and inference of molecularinteractions. We are releasing Atlas to the scientific com-munity in the hope that it will foster creative ideas for howto make novel associations between disparate sources ofdata using existing public data sets.Availability and requirementsAtlas is available from the UBC Bioinformatics Centre,University of British Columbia. The Atlas package can bedownloaded from the Atlas website at: http://bioinformatics.ubc.ca/atlas/The Atlas package contains the Atlas source code and rep-resents the core of the project. The package is distributedunder the GNU General Public License. Atlas is designedto run on Unix based systems. Please consult the usermanual (available with the distribution) for detailed con-figuration, compilation and installation instructions.Additional packages are also provided at the website listedabove. These packages include a snapshot of the NCBIC++ Toolkit (CVS version 20040505), a MySQL dump ofsample data, and additional documentation. The NCBIC++ Toolkit, that is provided, is required only for thoseusers who wish to build the loader applications or forthose that require the utilities that convert ASN.1 formatto GBFF, EMBL, and XML formats, etc. Those setting upthe database will need to install MySQL Server 4.x. Atlashas been tested, specifically, with MySQL Server versionsPage 14 of 16(page number not for citation purposes)the other. We are investigating the idea of an identifierconsolidation that can resolve mRNAs and proteins from4.0.9, 4.0.18 and 4.0.20, running on either Linux or AIX.BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/34The Atlas sequence-related binaries (toolbox applicationsand loader applications) are developed in C++ and there-fore a C++ compatible compiler, such as the one includedwith the GNU GCC suite of tools, should be installedbefore attempting to build these binaries. We have testedthe build process with GNU GCC versions 2.95.3, 2.96,3.1 and 3.2. In addition, MySQL Client version 4.x andparticularly its runtime library, libmysqlclient.a(so), isrequired. MySQL Client versions 4.0.14 and 4.1.0-alphawere tested. Details on the configuration and use of thislibrary are outlined, in more detail, in the Atlas manual.For users that require Atlas tools that are based on Java,such as the loading and retrieval tools for LocusLink,BIND, HPRD, and HomoloGene datasets, a compatibleJava interpreter must be installed. The API has been testedwith J2SE 1.4.1 and J2SE 1.4.2. The Atlas Java API alsorequires BioJava version 1.4pre, or higher.For those using the Perl based Atlas tools, a compatiblePerl interpreter must be installed. BioPerl version 1.4must also be installed. Perl version 5.6.1 has been tested.Each of the packages have their own minimum systemrequirements. Specific memory, hard disk space and CPUrequirements for each package are listed in the manual. Asa general guideline, it is essential to have a generousamount of available memory, especially if one anticipatesprocessing large sequences in memory. Another impor-tant factor is the amount of available hard disk space. Theamount of sequence data to be loaded into Atlas willlargely determine your disk space requirements. The Atlasdatabase requires a minimum of 50 GB (RefSeq), plusadequate space for satellite databases. The satellite data-bases include such things as GO, LocusLink, HPRD,BIND, MINT, and DIP, which are relatively smaller data-sets. Note that sequence data can greatly exceed these min-imum estimates and the requirements should be carefullyplanned.Authors' contributionsSS was the architect of the system, developed the C++ APIsand wrote the first draft of this manuscript. YH was thedatabase administrator responsible for schema design,data integrity and maintenance. TX contributed the JavaAPIs. MMSY contributed the PERL APIs. JL developed theC++ APIs, the toolbox and the user manual. BFFO was theprincipal investigator, conceived of the project and guidedits development. JL, YH, MSSY and BFFO all contributedto the writing of this manuscript.AcknowledgementsTX is supported by CIHR grant #MOP-53259, Juergen Kast and BFFO.We thank Michael E. Smoot, for his contributions and enhancements to his Templatized C++ Command Line Processor [51].Special thanks to Joanne Fox and Graeme Campbell for their help in review-ing and editing this paper.References1. Benson D, Karsch-Mizrachi I, Lipman D, Ostell J, Wheeler D: Gen-Bank: update. Nucleic Acids Res 2004:D23-26.2. Apweiler R, Bairoch A, Wu C, Barker W, Boeckmann B, Ferro S,Gasteiger E, Huang H, Lopez R, Magrane M, Martin M, Natale D,O'Donovan C, Redaschi N, Yeh L: UniProt: the Universal Pro-tein knowledgebase. Nucleic Acids Res 2004:115-119.3. Bader G, Hogue C: BIND-a data specification for storing anddescribing biomolecular interactions, molecular complexesand pathways. Bioinformatics 2000, 16(5):465-477.4. Bader G, Donaldson I, Wolting C, Ouellette B, Pawson T, Hogue C:BIND-The Biomolecular Interaction Network Database.Nucleic Acids Res 2001, 29:242-245.5. Bader G, Betel D, Hogue C: BIND: the Biomolecular InteractionNetwork Database. Nucleic Acids Res 2003, 31:248-250.6. Peri S, Navarro J, Kristiansen T, Amanchy R, Surendranath V, Muth-usamy B, Gandhi T, Chandrika K, Deshpande N, Suresh S, Rashmi B,Shanker K, Padma N, Niranjan V, Harsha H, Talreja N, VrushabendraB, Ramya M, Yatish A, Joy M, Shivashankar H, Kavitha M, Menezes M,Choudhury D, Ghosh N, Saravana R, Chandran S, Mohan S, Jonnala-gadda C, Prasad C, Kumar-Sinha C, Deshpande K, Pandey A: Humanprotein reference database as a discovery resource forproteomics. Nucleic Acids Res 2004:497-501.7. Gollub J, Ball C, Binkley G, Demeter J, Finkelstein D, Hebert J, Hern-andez-Boussard T, Jin H, Kaloper M, Matese J, Schroeder M, BrownP, Botstein D, Sherlock G: The Stanford Microarray Database:data access and quality assessment tools. Nucleic Acids Res 2003,31:94-96.8. Mootha V, Lepage P, Miller K, Bunkenborg J, Reich M, Hjerrild M, Del-monte T, Villeneuve A, Sladek R, Xu F, Mitchell G, Morin C, Mann M,Hudson T, Robinson B, Rioux J, Lander E: Identification of a genecausing human cytochrome c oxidase deficiency by integra-tive genomics. Proc Natl Acad Sci U S A 2003, 100(2):605-610.9. Stuart J, Segal E, Koller D, Kim S: A gene-coexpression networkfor global discovery of conserved genetic modules. Science2003, 302(5643):249-255.10. Sequence Retrieval System  [http://srs.embl-heidelberg.de:8000/]11. Wheeler D, Church D, Edgar R, Federhen S, Helmberg W, MaddenT, Pontius J, Schuler G, Schriml L, Sequeira E, Suzek T, Tatusova T,Wagner L: Database resources of the National Center for Bio-technology Information: update. Nucleic Acids Res 2004:35-40.12. Michalickova K, Bader G, Dumontier M, Lieu H, Betel D, Isserlin R,Hogue C: SeqHound: biological sequence and structure data-base as a platform for bioinformatics research. BMCBioinformatics 2002, 3:32-32.13. Kasprzyk A, Keefe D, Smedley D, London D, Spooner W, Melsopp C,Hammond M, Rocca-Serra P, Cox T, Birney E: EnsMart: a genericsystem for fast and flexible access to biological data. GenomeRes 2004, 14:160-169.14. Fujibuchi W, Goto S, Migimatsu H, Uchiyama I, Ogiwara A, AkiyamaY, Kanehisa M: DBGET/LinkDB: an integrated databaseretrieval system. Pacific Symp Biocomputing 1998:683-694.15. Huang Y, Ni T, Zhou L, Su S: JXP4BIGI: a generalized, Java XML-based approach for biological information gathering andintegration. Bioinformatics 2003, 19(18):2351-2358.16. Wilkinson M, Links M: BioMOBY: An open source biologicalweb services proposal. Briefings in Bioinformatics 2002,3(4):331-341.17. Stevens R, Baker P, Bechhofer S, Ng G, Jacoby A, Paton N, Goble C,Brass A: TAMBIS: transparent access to multiple bioinfor-matics information sources. Bioinformatics 2000, 16(2):184-185.18. Hermjakob H, Montecchi-Palazzi L, Lewington C, Mudali S, Kerrien S,Orchard S, Vingron M, Roechert B, Roepstorff P, Valencia A, MargalitH, Armstrong J, Bairoch A, Cesareni G, Sherman D, Apweiler R:IntAct – an open source molecular interaction database. NuclAcids Res 2004, 32:D452-D455.Page 15 of 16(page number not for citation purposes)We thank Miroslav Hatas and Graeme Campbell for systems administration support.19. Salwinski L, Miller C, Smith A, Pettit F, Bowie J, Eisenberg D: TheDatabase of Interacting Proteins: 2004 update. Nucleic AcidsRes 2004:449-451.Publish with BioMed Central   and  every scientist can read your work free of charge"BioMed Central will be the most significant development for disseminating the results of biomedical research in our lifetime."Sir Paul Nurse, Cancer Research UKYour research papers will be:available free of charge to the entire biomedical communitypeer reviewed and published immediately upon acceptancecited in PubMed and archived on PubMed Central BMC Bioinformatics 2005, 6:34 http://www.biomedcentral.com/1471-2105/6/3420. Online Mendelian Inheritance in Man, OMIM (TM). McKu-sick-Nathans Institute for Genetic Medicine, Johns HopkinsUniversity (Baltimore, MD) and National Center for Bio-technology Information, National Library of Medicine (Balti-more, MD) Bethesda, MD 2000 [http://www.ncbi.nlm.nih.gov/omim/].21. Pruitt K, Maglott D: RefSeq and LocusLink: NCBI gene-cen-tered resources. Nucleic Acids Res 2001, 29:137-140.22. Entrez Gene  [http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=gene]23. NCBI HomoloGene  [http://www.ncbi.nlm.nih.gov/HomoloGene/]24. NCBI Taxonomy  [http://www.ncbi.nlm.nih.gov/Taxonomy/]25. Ashburner M, Ball C, Blake J, Botstein D, Butler H, Cherry J, Davis A,Dolinski K, Dwight S, Eppig J, Harris M, Hill D, Issel-Tarver L, Kasar-skis A, Lewis S, Matese J, Richardson J, Ringwald M, Rubin G, SherlockG: Gene ontology: tool for the unification of biology. TheGene Ontology Consortium. Nat Genet 2000, 25:25-29.26. Harris M, Clark J, Ireland A, Lomax J, Ashburner M, Foulger R, EilbeckK, Lewis S, Marshall B, Mungall C, Richter J, Rubin G, Blake J, Bult C,Dolan M, Drabkin H, Eppig J, Hill D, Ni L, Ringwald M, BalakrishnanR, Cherry J, Christie K, Costanzo M, Dwight S, Engel S, Fisk D, Hir-schman J, Hong E, Nash R, Sethuraman A, Theesfeld C, Botstein D,Dolinski K, Feierbach B, Berardini T, Mundodi S, Rhee S, Apweiler R,Barrell D, Camon E, Dimmer E, Lee V, Chisholm R, Gaudet P, KibbeW, Kishore R, Schwarz E, Sternberg P, Gwinn M, Hannick L, Wort-man J, Berriman M, Wood V, de la Cruz N, Tonellato P, Jaiswal P, Sei-gfried T, White Ra: The Gene Ontology (GO) database andinformatics resource. Nucleic Acids Res 2004:258-261.27. Miyazaki S, Sugawara H, Ikeo K, Gojobori T, Tateno Y: DDBJ in thestream of various biological data. Nucleic Acids Res 2004:D31-34.28. Kulikova T, Aldebert P, Althorpe N, Baker W, Bates K, Browne P, vanden Broek A, Cochrane G, Duggan K, Eberhardt R, Faruque N, Gar-cia-Pastor M, Harte N, Kanz C, Leinonen R, Lin Q, Lombard V, LopezR, Mancuso R, McHale M, Nardone F, Silventoinen V, Stoehr P,Stoesser G, Tuli M, Tzouvara K, Vaughan R, Wu D, Zhu W, ApweilerR: The EMBL Nucleotide Sequence Database. Nucleic Acids Res2004:D27-30.29. MySQL  [http://www.mysql.com/]30. Hermjakob H, Montecchi-Palazzi L, Bader G, Wojcik J, Salwinski L,Ceol A, Moore S, Orchard S, Sarkans U, von Mering C, Roechert B,Poux S, Jung E, Mersch H, Kersey P, Lappe M, Li Y, Zeng R, Rana D,Nikolski M, Husi H, Brun C, Shanker K, Grant S, Sander C, Bork P,Zhu W, Pandey A, Brazma A, Jacq B, Vidal M, Sherman D, Legrain P,Cesareni G, Xenarios I, Eisenberg D, Steipe B, Hogue C, Apweiler R:The HUPO PSI's molecular interaction format-a communitystandard for the representation of protein interaction data.Nat Biotechnol 2004, 22(2):177-183.31. Open Biological Ontologies  [http://obo.sourceforge.net/]32. Sequence Ontology  [http://song.sourceforge.net/]33. Stoeckert C, Causton H, Ball C: Microarray databases: standardsand ontologies. Nature Genetics 2002, 32:469-473.34. Baker P, Goble C, Bechhofer S, Paton N, Stevens R, Brass A: Anontology for bioinformatics applications. Bioinformatics 1999,15(6):510-520.35. General Feature Format  [http://www.sanger.ac.uk/Software/formats/GFF/]36. Vakatov D, Siyan K, Ostell J, editors: The NCBI C++ Toolkit[Internet]. National Library of Medicine (US), National Center for Bio-technology Information, Bethesda (MD) 2003 [http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=Books].37. Wheeler D, Church D, Lash A, Leipe D, Madden T, Pontius J, SchulerG, Schriml L, Tatusova T, Wagner L, Rapp B: Database resourcesof the National Center for Biotechnology Information.Nucleic Acids Res 2001, 29:11-16.38. Shah S, He D, Sawkins J, Druce J, Quon G, Lett D, Zheng G, Xu T,Ouellette B: Pegasys: software for executing and integratinganalyses of biological sequences. BMC Bioinformatics 2004,5:40-40.39. Lewis S, Searle S, Harris N, Gibson M, Lyer V, Richter J, Wiel C,Bayraktaroglir L, Birney E, Crosby M, Kaminker J, Matthews B, Proch-nik S, Smithy C, Tupy J, Rubin G, Misra S, Mungall C, Clamp M:Apollo: a sequence annotation editor. Genome Biol 2002, 3(12):.40. Sequin  [http://www.ncbi.nlm.nih.gov/Sequin/]41. GAME XML DTD  [http://www.fruitfly.org/annot/gamexml.dtd.txt]43. Kitaura M, Suzuki N, Imai T, Takagi S, Suzuki R, Nakajima T, Hirai K,Nomiyama H, Yoshie O: Molecular cloning of a novel humanCC chemokine (Eotaxin-3) that is a functional ligand of CCchemokine receptor 3. J Biol Chem 1999, 274(39):27975-27980.44. Umland S, Wan Y, Shortall J, Shah H, Jakway J, Garlisi C, Tian F, EganR, Billah M: Receptor reserve analysis of the human CCR3receptor in eosinophils and CCR3-transfected cells. J LeukocBiol 2000, 67(3):441-447.45. Atlas website  [http://www.bioinformatics.ubc.ca/atlas/]46. Entrez  [http://www.ncbi.nlm.nih.gov/Entrez/index.html]47. Blueprint Initiative  [http://www.blueprint.org/]48. Karolchik D, Baertsch R, Diekhans M, Furey T, Hinrichs A, Lu Y,Roskin K, Schwartz M, Sugnet C, Thomas D, Weber R, Haussler D,Kent W: The UCSC Genome Browser Database. Nucleic AcidsRes 2003, 31:51-54.49. EMBL-EBI  [http://www.ebi.ac.uk/]50. The Wellcome Trust Sanger Institute  [http://www.sanger.ac.uk/]51. Smoot ME: Templatized C++ Command Line Parser Library.[http://tclap.sourceforge.net/].yours — you keep the copyrightSubmit your manuscript here:http://www.biomedcentral.com/info/publishing_adv.aspBioMedcentralPage 16 of 16(page number not for citation purposes)42. Submission of complete genomes or other large sequencerecords  [http://www.ncbi.nlm.nih.gov/Sequin/table.html]


Citation Scheme:


Citations by CSL (citeproc-js)

Usage Statistics



Customize your widget with the following options, then copy and paste the code below into the HTML of your page to embed this item in your website.
                            <div id="ubcOpenCollectionsWidgetDisplay">
                            <script id="ubcOpenCollectionsWidget"
                            async >
IIIF logo Our image viewer uses the IIIF 2.0 standard. To load this item in other compatible viewers, use this url:


Related Items