Overview
Welcome to the Open Collections API Documentation ! Below you'll find tutorials that will teach you how to use the Open Collections API and reference documentation for all its moving parts.
Where To Go
- If you haven't used an API before, head on over to API Basics. Which will get you started on accessing our data.
- If you have already used APIs and you just want to know how to get at our data, skip the basics and head straight on over to our Requests section and take a look a our Reference section. (You'll need an API key first.)
- Once you've made a request, you might need some help making sense of what it all means. In responses, we explore the structure of objects and the fields within those objects that the API returns.
- Now you should be ready to start building something with our API, however you might like to know the small print, so go check out our API's policies.
- Visit and read our terms of use page, before using the API.
- Reach us at library.opencollections@ubc.ca for inquiries. (or if you make something cool you'd like us to feature)
The Essentials
If you need to start straight away, here is how:
- https://oc-index.library.ubc.ca is the base URL of the Open Collections API.
- In order to pass requests to the API, you'll need to register an API key first. You also need to append &api_key=[:YOUR_API_KEY] (where [:YOUR_API_KEY] is the API key you received after registering for the Open Collections API Key) to any requests you make.
- All records are returned by default as JSON-LD objects however you can request other formats by as descripted over at output formats
- We've got lots of collections & fields you can query, and a whole lot of ways you can query them.
- The main resources the API returns are Collection Metadata and Item Metadata, for Image Metadata or image downloads you will need to check out our IIIF documentation (Media API for audio & video coming soon)
API
API Basics
What is an API?
At its most basic, an API, or Application Programming Interface, is made up of a set of defined methods that someone can use to communicate with a (often complex) software system, and get back responses in a way that a computer (and, with some practice, a human) can understand.
In essence, an API defines the 'language' a system speaks. Like a language, it has its own vocabulary with terms that have special meanings (e.g., property names and labels), grammar (how those property names and labels are arranged collectively, its schema), and syntax (i.e., how the information itself is arranged). Like a language, you can use it to ask questions and understand responses. And like a language, with a little practice, it can be a powerful and extensible tool for communication.
A request is a URL sent to the web server over HTTP with the expectation of getting resource items back in the form of human-readable text or data. The URL supplies the web server with everything it needs to create and return a correct response. This is called a RESTful approach to API design and is employed by the Open Collections API.
How do I use it?
To get started using the Open Collections API:
- Register an API Key
- Use your web browser's URL bar or a browser plugin like Postman to make a request
- Read the response
API Keys
To control abuse, users will need to register their email address to receive an API Key (much like DPLA).
Users are also constrained to no more than 200 requests per minute.
| Name | API Key | Rate Limit: |
|---|---|---|
| Public API Key | ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c | 10req/min |
| Your API Key | You haven't yet registered an API key, instructions on how you can are below. | 200req/min |
Registering an API Key
Register your email address with us to receive an API key.
API Key:
An email has been sent to the address entered. Usage rates are limited until the address has been confirmed.
Requests
The base URL of the API is: https://oc-index.library.ubc.ca
Resource Types
When you formulate a REST query, you have to decide which resource type you want to access. The two resource types currently offered are described below.
Collection Metadata
A collection is a reference to a collection of multiple items indexed in Open Collections. Collections have two RESTful routes associated with them, examples are provided below;
URL Structure
The RESTful URL to request a collections metadata is structured as follows:
https://oc-index.library.ubc.ca/collections/ [:name]
Where [:name] is the nickname of a collection in Open Collections. See all the nicknames and descriptions of our collections at 'Collection Reference'
The RESTful URL to request a collections metadata is structured as follows:
https://oc-index.library.ubc.ca/collections/ [:name]
Where [:name] is the nickname of a collection in Open Collections. See all the nicknames and descriptions of our collections at 'Collection Reference'
Collection Metadata Example
For your convenience here is an example of a call to get the collection berkpost's metadata.
https://oc-index.library.ubc.ca/collections/berkpost
For your convenience here is an example of a call to get the collection berkpost's metadata.
https://oc-index.library.ubc.ca/collections/berkpost
Collection Items List
The collections item list will allow you to traverse through all of a collections items. For larger collections we recommend you use the URL options ?limit=[:limit] and &offset=[:offset] to prevent slow load times and possible browser crashing.
URL Structure
The RESTful URL to request a collections items is structured as follows:
https://oc-index.library.ubc.ca/collections/ [:name] /items
Where :name is the nickname of a collection in Open Collections.
Additional URL Paramaters
?limit - Set the limit of items to return
&offset - Set the offset of items to start from
The RESTful URL to request a collections items is structured as follows:
https://oc-index.library.ubc.ca/collections/ [:name] /items
Where :name is the nickname of a collection in Open Collections.
Additional URL Paramaters
?limit - Set the limit of items to return
&offset - Set the offset of items to start from
Collections Item List Example
For your convenience here is an example of a call to get all of the collection berkpost's items.
https://oc-index.library.ubc.ca /collections/berkpost/items
For your convenience here is an example of a call to get all of the collection berkpost's items.
https://oc-index.library.ubc.ca /collections/berkpost/items
Item Metadata
An item is a reference to a single piece of content indexed in Open Collections. The content can be, for example, a book, an image, a video, etc. The content is digitized from its original, physical source and uploaded to an online repository. After retrieving items, developers can display or follow links to their original online digital records.
Structure
The RESTful URL to request a collections items is structured as follows:
https://oc-index.library.ubc.ca/collections/ [:name] /items/ [:itemId]
Where [:name] is the nickname of a collection in Open Collections and [:itemId] is the items identifier.
The RESTful URL to request a collections items is structured as follows:
https://oc-index.library.ubc.ca/collections/ [:name] /items/ [:itemId]
Where [:name] is the nickname of a collection in Open Collections and [:itemId] is the items identifier.
Single Item Example
For your convenience here is an example of a call to get a single item from the berkpost collection.
https://oc-index.library.ubc.ca /collections/berkpost/items/1.0001623
For your convenience here is an example of a call to get a single item from the berkpost collection.
https://oc-index.library.ubc.ca /collections/berkpost/items/1.0001623
Advanced Usage
More advanced users that are looking to really dive into our API should check out the following Elastic Search documentation:
Reference
Endpoints
The following API endpoints are available for use
| Method | Endpoint | Description | Status |
|---|---|---|---|
| GET | collections | Returns a list of all collections | Online |
| GET | collections/:collection_identifier | Returns the metadata for a specific collection | Online |
| GET | collections/:collection_identifier/download/collection | Returns a GZIP download of a specific collection's metadata | Online |
| GET | collections/:collection_identifier/items | Returns a list of all items for a specific collection | Online |
| GET | collections/:collection_identifier/items/:item_identifier | Returns the metadata for a specific collection's single item | Online |
| GET | collections/:collection_identifier/_total | Returns the item count of a specific collection | Online |
| GET | collections/:collection_identifier/linked | Returns JSON-LD collection metadata for a specific collection | Offline |
| GET | collections/:collection_identifier/items/:item_identifier/linked | Returns JSON-LD item metadata for a specific collection | Offline |
| POST | search | Returns a specific collections linked json | Online |
Collections
| Collection ID | Collection Name |
|---|---|
| 59371 | American Physical Society Northwestern Section Annual Meeting (APS-NW) (11th : 2009) |
| artefacts | Ancient Artefacts |
| mccormick | Andrew McCormick Maps and Prints |
| 13139 | Annual Forestry Lecture in Sustainability |
| 11568 | Asian Library: Articles/Publications |
| 11569 | Asian Library: Audiovisual Records |
| 11571 | Asian Library: Miscellaneous |
| 11570 | Asian Library: News Articles |
| 33850 | Award: UBC Library Innovative Dissemination of Research Award |
| bcbooks | BC Historical Books |
| bcsessional | BC Sessional Papers |
| 48630 | BIRS Workshop Lecture Videos |
| berkpost | Berkeley 1968-1973 Poster Collection |
| biblos | Biblos |
| bch | British Columbia History |
| 59367 | British Columbia Mine Reclamation Symposium |
| 26856 | CWGS Lecture Series: Podcasts and Notes |
| 59370 | Canadian Summer School on Quantum Information (CSSQI) (10th : 2010) |
| capilano | Capilano Timber Company Fonds |
| darwin | Charles Darwin Letters |
| 33067 | Chinese Canadian Stories media |
| chungex | Chung Exhibit Items |
| chungosgr | Chung Oversize and Graphic Materials |
| chungphotos | Chung Photographs |
| chungpub | Chung Published Works |
| chungtext | Chung Textual Materials |
| citraudio | CiTR Audiotapes |
| 59404 | Congress of the Humanities and Social Sciences (77th : 2008) |
| 46624 | Consortium for Nursing History Inquiry |
| cg | Creative Giving |
| davidconde | David Conde Fonds |
| davidsonia | Davidsonia |
| delgamuukw | Delgamuukw Trial Transcripts |
| 55474 | Digital Library Federation (DLF) (2015) |
| discorder | Discorder |
| dorothyburn | Dorothy Burnett Bookbinding Tools |
| djcox | Doug and Joyce Cox Research Collection |
| 801 | E. W. Vogt Science Symposium |
| 24 | Electronic Theses and Dissertations (ETDs) 2008+ |
| ecrosby | Emma Crosby Letters |
| squeezes | Epigraphic Squeezes |
| etheljohns | Ethel Johns Fonds |
| 34295 | Explorations and Education |
| focus | FOCUS |
| 52383 | Faculty Research and Publications |
| first100theses | First Hundred Theses |
| fisherman | Fisherman Publishing Society Collection |
| florence | Florence Nightingale Letters |
| 40947 | Graduate Law Students' Conference Publications |
| 42591 | Graduate Research [non-thesis] |
| 59757 | Grand Rounds Seminar Series |
| gvrdmaps | Greater Vancouver Regional District Planning Department Land Use Maps |
| bullock | H. Bullock-Webster fonds |
| hawthorn | Hawthorn Fly Fishing & Angling Collection |
| nursing | History of Nursing |
| iel | Indian Education Newsletter |
| feeders | Infant Feeders Collection |
| 59405 | Insights from Nuclear Magnetic Resonance (NMR) : A symposium in honour of Myer Bloom (1993) |
| 59585 | International Conference of Institutes and Libraries for Chinese Overseas Studies (WCILCOS) (5th : 2012) |
| 53032 | International Conference on Applications of Statistics and Probability in Civil Engineering (ICASP) (12th : 2015) |
| 52657 | International Conference on Engineering Education for Sustainable Development (EESD) (7th : 2015) |
| 59278 | International Conference on Gas Hydrates (ICGH) (6th : 2008) |
| 53926 | International Conference on Health Promoting Universities and Colleges (7th : 2015) |
| 52660 | International Construction Specialty Conference of the Canadian Society for Civil Engineering (ICSC) (5th : 2015) |
| 62152 | International Institute for Critical Studies in Improvisation (IICSI) Colloquium |
| 59406 | Investigating Our Practices (IOP) |
| 27306 | Irving K. Barber Learning Centre Events |
| jphotos | Japanese Canadian Photograph Collection |
| tokugawa | Japanese Maps of the Tokugawa Era |
| johnkeenlyside | John Keenlyside Legal Research Collection |
| kinesis | Kinesis |
| holland | Laura Holland Fonds |
| 50555 | Leader in Residence |
| 23304 | Library Assessment: LibQUAL Surveys |
| 23303 | Library Assessment: Publications/Presentations |
| 23302 | Library Assessment: Statistics |
| 2689 | Library Events |
| 23553 | Library Research Reports |
| 494 | Library Staff Papers and Presentations |
| creelman | Lyle Creelman Fonds |
| macmillan | MacMillan Bloedel Limited fonds |
| 59369 | Metropolis British Columbia Policy Research Symposium (MBC) (2008) |
| upubmisc | Miscellaneous Documents |
| 42446 | Multidisciplinary Undergraduate Research Conference (MURC), 2012+ |
| 60499 | NEXUS Spring Institute |
| agassiz | Newspapers - Agassiz Record |
| xalberniadv | Newspapers - Alberni Advocate |
| armstrongad | Newspapers - Armstrong Advance |
| arlaadvo | Newspapers - Arrow Lake Advocate |
| bcln | Newspapers - BC Labor News (Vancouver) |
| bclumber | Newspapers - BC Lumberman (Vancouver) |
| bctu | Newspapers - BC Trades Unionist (Vancouver) |
| xbellacoo | Newspapers - Bella Coola Courier |
| bensun | Newspapers - Bennett Sun |
| bcfed | Newspapers - British Columbia Federationist |
| bcnews | Newspapers - British Columbia News |
| xbcrecord | Newspapers - British Columbia Record |
| bcret | Newspapers - British Columbia Retailer |
| bct | Newspapers - British Columbia Tribune (Yale) |
| brooklynnews | Newspapers - Brooklyn News |
| cflacla | Newspapers - Canadian Farmer Labor Advocate and Canadian Labor Advocate |
| canford | Newspapers - Canford Radium |
| cascade | Newspapers - Cascade Record |
| cassiarnews | Newspapers - Cassiar News (Stewart) |
| chasetrib | Newspapers - Chase Tribune |
| chilliwackfp | Newspapers - Chilliwack Free Press |
| coalmont | Newspapers - Coalmont Courier |
| coasmine | Newspapers - Coast Miner (Van Anda) |
| columbiarev | Newspapers - Columbia Review |
| courtenayrev | Newspapers - Courtenay Review |
| cwn | Newspapers - Courtenay Weekly News |
| cranherald | Newspapers - Cranbrook Herald |
| croftongaz | Newspapers - Crofton Gazette and Cowichan News |
| cumberlandis | Newspapers - Cumberland Islander |
| dbc | Newspapers - Daily British Columbian |
| xdbr | Newspapers - Daily Building Record |
| enterprise | Newspapers - Duncan Enterprise |
| eastkootmine | Newspapers - East Kootenay Miner (Golden) |
| echo | Newspapers - Echo (Duncan) |
| eveningt | Newspapers - Evening Telegraph (Victoria) |
| evewoross | Newspapers - Evening World (Rossland) |
| fgherald | Newspapers - Fort George Herald |
| fraseradvanc | Newspapers - Fraser Advance (Chilliwack) |
| gcdb | Newspapers - General Conference Daily Bulletin (Victoria) |
| glennews | Newspapers - Glenora News |
| goldenera | Newspapers - Golden Era |
| goldentimes | Newspapers - Golden Times |
| gvchinook | Newspapers - Greater Vancouver Chinook |
| hqueek | Newspapers - Hazelton Queek |
| xhotsprings | Newspapers - Hot Springs News (Ainsworth) |
| indworld | Newspapers - Industrial World (Rossland) |
| kwawa | Newspapers - Kamloops Wawa |
| xkelownarec | Newspapers - Kelowna Record and The Orchard City Record |
| kerechro | Newspapers - Keremeos Chronicle |
| kootstar | Newspapers - Kootenay Star (Revelstoke) |
| laborstar | Newspapers - Labor Star (Vancouver) |
| ardeau | Newspapers - Lardeau Eagle (Ferguson) |
| lardeaum | Newspapers - Lardeau Mining Review (Trout Lake) |
| leadera | Newspapers - Leader Advocate (Vancouver) |
| locla | Newspapers - Lowery's Claim |
| mherald | Newspapers - Mail Herald (Revelstoke) |
| michelr | Newspapers - Michel Recorder |
| mpadvocate | Newspapers - Mount Pleasant Advocate (Vancouver) |
| ndaymine | Newspapers - Nelson Daily Miner |
| nwminer | Newspapers - Nelson Weekly Miner |
| nwdn | Newspapers - New Westminster Daily News |
| newestimes | Newspapers - New Westminster Times |
| nicoheral | Newspapers - Nicola Herald |
| norcoa | Newspapers - North Coast (Port Simpson) |
| omr | Newspapers - Okanagan Mining Review |
| omineca | Newspapers - Omineca Herald (Hazelton) |
| ominecaminer | Newspapers - Omineca Miner (Hazelton) |
| paccannw | Newspapers - Pacific Canadian (New Westminster) |
| paystreak | Newspapers - Paystreak (Sandon) |
| peloyalist | Newspapers - Port Essington Loyalist |
| pmgazette | Newspapers - Port Moody Gazette |
| prj | Newspapers - Prince Rupert Journal |
| princero | Newspapers - Prince Rupert Optimist |
| pwv | Newspapers - Progress and Week – Victoria |
| cranbrookpro | Newspapers - Prospector (Cranbrook) |
| proslill | Newspapers - Prospector (Lillooet) |
| prossross | Newspapers - Prospector (Rossland) |
| qcminer | Newspapers - Quartz Creek Miner |
| qcislander | Newspapers - Queen Charlotte Islander |
| redflag | Newspapers - Red Flag (Vancouver) |
| xrevherald | Newspapers - Revelstoke Herald |
| sfjcbce | Newspapers - San Francisco Journal of Commerce B.C. Edition (Victoria) |
| satworld | Newspapers - Saturday World (Rossland) |
| silsil | Newspapers - Silverton Silvertonian |
| slodrill | Newspapers - Slocan Drill |
| smreview | Newspapers - Slocan Mining Review |
| slocanp | Newspapers - Slocan Prospector |
| slorec | Newspapers - Slocan Record |
| surreytimes | Newspapers - Surrey Times (Cloverdale) |
| xabpost | Newspapers - The Abbotsford Post |
| advance | Newspapers - The Advance (Midway) |
| xanaconda | Newspapers - The Anaconda News |
| xatlin | Newspapers - The Atlin Claim |
| xboundarycr | Newspapers - The Boundary Creek Times |
| xcariboosen | Newspapers - The Cariboo Sentinel (Barkerville) |
| xcoastnews | Newspapers - The Coast News (Gibsons) |
| xcrestonrev | Newspapers - The Creston Review |
| xcumberland | Newspapers - The Cumberland News |
| dcanadi | Newspapers - The Daily Canadian |
| xdailyledg | Newspapers - The Daily Ledger (Ladysmith) |
| daytele | Newspapers - The Daily Telegram |
| deltnews | Newspapers - The Delta News |
| delttime | Newspapers - The Delta Times |
| despatch | Newspapers - The Despatch (Morrissey) |
| disledfer | Newspapers - The District Ledger (Fernie) |
| xenderby | Newspapers - The Enderby Press and Walker's Weekly |
| evenkoot | Newspapers - The Evening Kootenaian |
| expressnv | Newspapers - The Express (North Vancouver) |
| fernieled | Newspapers - The Fernie Ledger (Fernie) |
| gfminer | Newspapers - The Grand Forks Miner |
| xgrandforks | Newspapers - The Grand Forks Sun |
| greemine | Newspapers - The Greenwood Miner |
| xhedley | Newspapers - The Hedley Gazette |
| htimes | Newspapers - The Hosmer Times |
| xindependen | Newspapers - The Independent (Vancouver) |
| koolib | Newspapers - The Kootenay Liberal |
| xkootmail | Newspapers - The Kootenay Mail |
| ledgefern | Newspapers - The Ledge (Fernie) |
| xledgreen | Newspapers - The Ledge (Greenwood) |
| ledge | Newspapers - The Ledge (Nakusp) |
| ledgenel | Newspapers - The Ledge (Nelson) |
| xnakledge | Newspapers - The Ledge (New Denver) |
| lilladva | Newspapers - The Lillooet Advance |
| marytrib | Newspapers - The Marysville Tribune |
| xmassett | Newspapers - The Massett Leader |
| xminer | Newspapers - The Miner (Nelson) |
| xminingrev | Newspapers - The Mining Review (Sandon) |
| misscity | Newspapers - The Mission City News |
| mmention | Newspapers - The Morrissey Mention |
| mminer | Newspapers - The Morrissey Miner |
| xmoyie | Newspapers - The Moyie Leader |
| nanacour | Newspapers - The Nanaimo Courier |
| nanamail | Newspapers - The Nanaimo Mail |
| xnelsonecon | Newspapers - The Nelson Economist |
| xnicola | Newspapers - The Nicola Valley News (Merritt) |
| thenugget | Newspapers - The Nugget |
| xpentimes | Newspapers - The Peninsula Times |
| penpress | Newspapers - The Penticton Press |
| xphoenix | Newspapers - The Phoenix Pioneer |
| xprospector | Newspapers - The Prospector (Fort Steele) |
| thestar | Newspapers - The Star (Port Essington) |
| thesun | Newspapers - The Sun (Port Essington) |
| xtribune | Newspapers - The Tribune (Nelson) |
| thewave | Newspapers - The Wave (Victoria) |
| xwestcall | Newspapers - The Western Call (Vancouver) |
| vslp | Newspapers - Valley Sentinel - Langley Prairie |
| vanbuildrec | Newspapers - Vancouver Building Record |
| beaverdell | Newspapers - West Forks News (Beaverdell) |
| wclarion | Newspapers - Western Clarion (Vancouver) |
| westho | Newspapers - Westward Ho! |
| ymirherald | Newspapers - Ymir Herald |
| ymirminer | Newspapers - Ymir Miner |
| ymirmirror | Newspapers - Ymir Mirror |
| ohs | Okanagan Historical Society Reports |
| hundred | One Hundred Poets | 百人一首 |
| 46718 | PIMS Newsletter |
| 46721 | PIMS Year in Review |
| prism | PRISM international |
| 33066 | Perspectives Newspaper |
| anderson | Peter Anderson fonds |
| presrep | Presidents’ Reports |
| asian | Puban (蒲坂藏書) & Pang Jingtang (龐鏡塘藏書) Collections |
| mathison | R. Mathison Collection |
| bookplate | RBSC Bookplates |
| 42020 | RBSC: Audiovisual Records |
| libsenrep | Report of the University Librarian to the Senate |
| 831 | Retrospective Theses and Dissertations, 1919-2007 |
| rosetti | Rosetti Studios - Stanley Park Collection |
| royalfisk | Royal Fisk Gold Rush Letters |
| saga | SAGA Document Collection |
| 310 | SCARP Graduating Projects |
| 51869 | Science One Research Projects |
| senmin | Senate Minutes |
| 33426 | Supplementary Thesis Materials and Errata |
| 59374 | Symposium on Early Modern Japanese Values and Individuality (2013) |
| 51833 | TRIUMF Reports |
| 59368 | Tailings and Mine Waste Conference |
| tairikunipp | Tairiku Nippo (Continental Daily News) |
| alumchron | The Graduate Chronicle/The UBC Alumni Chronicle/Trek |
| touchpoints | Touchpoints |
| tgdp | Traité général des pesches |
| arkley | Tremaine Arkley Croquet Collection |
| archivesav | UBC Archives Audio Recordings Collection |
| arphotos | UBC Archives Photograph Collection |
| 25332 | UBC Authors and Their Works Program, 1991-2006 |
| calendars | UBC Calendars |
| 52387 | UBC Community and Partner Publications |
| 31776 | UBC Faculty Publications Lists (1928-1969) |
| 31775 | UBC Historical Sound and Moving Image Collection |
| ubchist | UBC History |
| fisheries | UBC Institute of Fisheries Field Records |
| 43377 | UBC Japanese Canadian Students of 1942 |
| 33755 | UBC Law Special Lectures |
| ubcavfrc | UBC Legacy Video Collection |
| specialp | UBC Library Digitization Centre Special Projects |
| framed | UBC Library Framed Works Collection |
| ubclibnews | UBC Library News |
| ubcmedicine | UBC Medicine |
| abacusopen | UBC Open Data |
| 53169 | UBC President's Speeches and Writings |
| 641 | UBC Press Publications, Supplements, and Catalogues |
| ubcreports | UBC Reports |
| 18861 | UBC Social Ecological Economic Development Studies (SEEDS) Student Reports |
| ubcstuhan | UBC Student Handbooks |
| ubctp | UBC Theatre Programmes |
| ubcyearb | UBC Yearbooks |
| ubysseynews | Ubyssey |
| 52966 | Undergraduate Research |
| ubclsmm | University Publications - UBC Library Staff Meeting Minutes |
| langmann | Uno Langmann Family Collection of British Columbia Photographs |
| 12708 | Vancouver Institute Lectures |
| vma | Vancouver Medical Association |
| 43962 | Verna J. Kirkness - Speeches |
| wwposters | WWI & WWII Posters |
| manuscripts | Western Manuscripts And Early Printed Books |
| westland | Westland |
| 58233 | Workshop for Instruction in Library Use (WILU) (45th : 2016) |
| wwiphoto | World War I British press photograph collection |
| 32618 | Xwi7xwa Library: Aboriginal Education |
| 44311 | Xwi7xwa Library: First Nations House of Learning Collections |
| 29962 | Xwi7xwa Library: Indigenous Librarianship / Indigenous Knowledge Organization |
| 29912 | Xwi7xwa Library: Publications |
| yipsang | Yip Sang Collection |
| 33381 | cIRcle License Text |
| 32457 | iSchool @ UBC : School of Library, Archival and Information Studies (SLAIS) Research Days |
| the432 | 432 |
Fields
To view more about our metadata terms please view this page.
Output Formats
Responses from the API are returned in JSON by default however there are a number of different formats available which we will show you how to request for below.
GET Requests
For GET requests you should replace
:outputFormat
in the URL below with the format you want returned:
https://oc-index.library.ubc.ca/collections/
[:name]
/items/
[:itemId]
/output-format/
[:outputFormat]
POST Requests
For POST requests you should set the HTTP Accept header to the format you want returned.
Available Formats
| Description | GET Value | POST Value |
|---|---|---|
| Json conforming to ubc metadata manual | json | application/json |
| Json-ld (with ubc property tags) | ubc+json | application/json |
| Json-ld | ld | application/json |
| Json-ld | ld+json | application/json |
| Rdf/xml from json-ld (deprecated) | ld+rdf | application/xml |
| Rdf+json rdf in json | rdf+json | application/xml |
| Rdf/xml | rdf | application/xml |
| Rdf/xml | xml | application/xml |
| Xml stored at datacite (the metadata attached to the document doi) | datacite | application/xml |
| Turtle stored at datacite (the metadata attached to the document doi) | dx | text/plain |
| Turtle stored at datacite (the metadata attached to the document doi) | dx+turtle | text/plain |
| Bibtex stored at datacite (the metadata attached to the document doi) | dx+bibtex | text/plain |
| Ris stored at datacite (the metadata attached to the document doi) | dx+ris | text/plain |
| Rdf stored at datacite (the metadata attached to the document doi) | dx+rdf | application/xml |
| Plain text (turtle) | turtle | text/plain |
| Plain text (ntriples) | ntriples | text/plain |
| Just the item's fulltext (gosh all that full text!!) | plaintext | text/plain |
IIIF
To access our images programmatically you will need to learn about the International Image Interoperability Framework.
The URL of the IIIF server is: https://iiif.library.ubc.ca
Note:
One thing you will need to remember when working with results from the Open Collections API and using the IIIF API is that item identifiers recieved from Open Collections need to have the period replaced with a dash to work with the IIIF API. For instance '1.0001258' would become '1-0001258'
One thing you will need to remember when working with results from the Open Collections API and using the IIIF API is that item identifiers recieved from Open Collections need to have the period replaced with a dash to work with the IIIF API. For instance '1.0001258' would become '1-0001258'
Example
So say we find this wonderful item about winning the war by eating less bread and want to view the item in IIIF.
First you'll need to look at the 'Embed' accordion on the item's page and if you open that up you will see the following link: http://iiif.library.ubc.ca/presentation/cdm.wwposters.1-0038290/manifest
This will give us the manifest for the image/s in JSON, which includes the metadata and information about the sequences(image/s) like so:
{
"label": "The Kitchen is the [key] to victory. Eat less bread",
"viewingDirection": "left-to-right",
"viewingHint": "paged",
"metadata": [
{
"label": "Collection",
"value": [
{
"label": "Is Part Of",
"value": "World War I Poster and Broadside Collection",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/isPartOf",
"classmap": "dpla:SourceResource",
"property": "dcterms:isPartOf"
},
"iri": "http:\/\/purl.org\/dc\/terms\/isPartOf",
"explain": "A Dublin Core Terms Property; A related resource in which the described resource is physically or logically included."
}
]
},
{
"label": "DateAvailable",
"value": [
{
"label": "Date Available",
"value": "2013-10",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/issued",
"classmap": "edm:WebResource",
"property": "dcterms:issued"
},
"iri": "http:\/\/purl.org\/dc\/terms\/issued",
"explain": "A Dublin Core Terms Property; Date of formal issuance (e.g., publication) of the resource."
}
]
},
{
"label": "DateCreated",
"value": [
{
"label": "Date Created",
"value": "[between 1914 and 1918?]",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/created",
"classmap": "oc:SourceResource",
"property": "dcterms:created"
},
"iri": "http:\/\/purl.org\/dc\/terms\/created",
"explain": "A Dublin Core Terms Property; Date of creation of the resource."
}
]
},
{
"label": "Description",
"value": [
{
"label": "Description",
"value": "World War I poster.",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/description",
"classmap": "dpla:SourceResource",
"property": "dcterms:description"
},
"iri": "http:\/\/purl.org\/dc\/terms\/description",
"explain": "A Dublin Core Terms Property; An account of the resource.; Description may include but is not limited to: an abstract, a table of contents, a graphical representation, or a free-text account of the resource."
}
]
},
{
"label": "DigitalResourceOriginalRecord",
"value": [
{
"label": "Digital Resource Original Record",
"value": "https:\/\/open.library.ubc.ca\/collections\/wwposters\/items\/1.0038290",
"attrs": {
"lang": "en",
"ns": "http:\/\/www.europeana.eu\/schemas\/edm\/aggregatedCHO",
"classmap": "ore:Aggregation",
"property": "edm:aggregatedCHO"
},
"iri": "http:\/\/www.europeana.eu\/schemas\/edm\/aggregatedCHO",
"explain": "A Europeana Data Model Property; The identifier of the source object, e.g. the Mona Lisa itself. This could be a full linked open date URI or an internal identifier"
}
]
},
{
"label": "Extent",
"value": [
{
"label": "Extent",
"value": "1 poster ; 76 x 51 cm",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/extent",
"classmap": "dpla:SourceResource",
"property": "dcterms:extent"
},
"iri": "http:\/\/purl.org\/dc\/terms\/extent",
"explain": "A Dublin Core Terms Property; The size or duration of the resource."
}
]
},
{
"label": "FileFormat",
"value": [
{
"label": "Format",
"value": "image\/jpeg",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/elements\/1.1\/format",
"classmap": "edm:WebResource",
"property": "dc:format"
},
"iri": "http:\/\/purl.org\/dc\/elements\/1.1\/format",
"explain": "A Dublin Core Elements Property; The file format, physical medium, or dimensions of the resource.; Examples of dimensions include size and duration. Recommended best practice is to use a controlled vocabulary such as the list of Internet Media Types [MIME]."
}
]
},
{
"label": "Genre",
"value": [
{
"label": "Genre",
"value": "Posters",
"attrs": {
"lang": "en",
"ns": "http:\/\/www.europeana.eu\/schemas\/edm\/hasType",
"classmap": "dpla:SourceResource",
"property": "edm:hasType"
},
"iri": "http:\/\/www.europeana.eu\/schemas\/edm\/hasType",
"explain": "A Europeana Data Model Property; This property relates a resource with the concepts it belongs to in a suitable type system such as MIME or any thesaurus that captures categories of objects in a given field. It does NOT capture aboutness"
}
]
},
{
"label": "Identifier",
"value": [
{
"label": "UBC Call Number",
"value": "SPAM462C",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/identifier",
"classmap": "dpla:SourceResource",
"property": "dcterms:identifier"
},
"iri": "http:\/\/purl.org\/dc\/terms\/identifier",
"explain": "A Dublin Core Terms Property; An unambiguous reference to the resource within a given context.; Recommended best practice is to identify the resource by means of a string conforming to a formal identification system."
}
]
},
{
"label": "IsShownAt",
"value": [
{
"label": "DOI",
"value": "10.14288\/1.0038290",
"attrs": {
"lang": "en",
"ns": "http:\/\/www.europeana.eu\/schemas\/edm\/isShownAt",
"classmap": "edm:WebResource",
"property": "edm:isShownAt"
},
"iri": "http:\/\/www.europeana.eu\/schemas\/edm\/isShownAt",
"explain": "A Europeana Data Model Property; An unambiguous URL reference to the digital object on the provider\u2019s website in its full information context."
}
]
},
{
"label": "Language",
"value": [
{
"label": "Language",
"value": "English",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/language",
"classmap": "dpla:SourceResource",
"property": "dcterms:language"
},
"iri": "http:\/\/purl.org\/dc\/terms\/language",
"explain": "A Dublin Core Terms Property; A language of the resource.; Recommended best practice is to use a controlled vocabulary such as RFC 4646 [RFC4646]."
}
]
},
{
"label": "Notes",
"value": [
{
"label": "Notes",
"value": "Printed on bottom left: \"F.C. no. 22.\" Donated by Simon Newcomb.",
"attrs": {
"lang": "en",
"ns": "http:\/\/www.w3.org\/2009\/08\/skos-reference\/skos.html#note",
"classmap": "skos:Concept",
"property": "skos:note"
},
"iri": "http:\/\/www.w3.org\/2009\/08\/skos-reference\/skos.html#note",
"explain": "Simple Knowledge Organisation System; Notes are used to provide information relating to SKOS concepts. There is no restriction on the nature of this information, e.g., it could be plain text, hypertext, or an image; it could be a definition, information about the scope of a concept, editorial information, or any other type of information."
}
]
},
{
"label": "Publisher",
"value": [
{
"label": "Publisher - Original",
"value": "London, England : Printed by Hazell, Watson & Viney Ltd. Litho.",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/publisher",
"classmap": "dpla:SourceResource",
"property": "dcterms:publisher"
},
"iri": "http:\/\/purl.org\/dc\/terms\/publisher",
"explain": "A Dublin Core Terms Property; An entity responsible for making the resource available.; Examples of a Publisher include a person, an organization, or a service."
},
{
"label": "Publisher - Digital",
"value": "Vancouver : University of British Columbia Library",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/publisher",
"classmap": "dpla:SourceResource",
"property": "dcterms:publisher"
},
"iri": "http:\/\/purl.org\/dc\/terms\/publisher",
"explain": "A Dublin Core Terms Property; An entity responsible for making the resource available.; Examples of a Publisher include a person, an organization, or a service."
}
]
},
{
"label": "Rights",
"value": [
{
"label": "Rights",
"value": "Images provided for research and reference use only. Permission to publish, copy, or otherwise use these images must be obtained from Rare Books and Special Collections: http:\/\/rbsc.library.ubc.ca",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/rights",
"classmap": "edm:WebResource",
"property": "dcterms:rights"
},
"iri": "http:\/\/purl.org\/dc\/terms\/rights",
"explain": "A Dublin Core Terms Property; Information about rights held in and over the resource.; Typically, rights information includes a statement about various property rights associated with the resource, including intellectual property rights."
}
]
},
{
"label": "SortDate",
"value": [
{
"label": "Sort Date",
"value": "1918",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/elements\/1.1\/date",
"classmap": "dpla:SourceResource"
},
"iri": "http:\/\/purl.org\/dc\/elements\/1.1\/date",
"explain": "A Dublin Core Elements Property; A point or period of time associated with an event in the lifecycle of the resource.; Date may be used to express temporal information at any level of granularity. Recommended best practice is to use an encoding scheme, such as the W3CDTF profile of ISO 8601 [W3CDTF]."
}
]
},
{
"label": "Source",
"value": [
{
"label": "Source",
"value": "Original Format: University of British Columbia. Library. Rare Books and Special Collections. World War I Poster and Broadside Collection. SPAM462C",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/source",
"classmap": "oc:SourceResource",
"property": "dcterms:source"
},
"iri": "http:\/\/purl.org\/dc\/terms\/source",
"explain": "A Dublin Core Terms Property; A related resource from which the described resource is derived.; The described resource may be derived from the related resource in whole or in part. Recommended best practice is to identify the related resource by means of a string conforming to a formal identification system."
}
]
},
{
"label": "Title",
"value": [
{
"label": "Title",
"value": "The Kitchen is the [key] to victory. Eat less bread",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/title",
"classmap": "dpla:SourceResource",
"property": "dcterms:title"
},
"iri": "http:\/\/purl.org\/dc\/terms\/title",
"explain": "A Dublin Core Terms Property; TA name given to the resource."
}
]
},
{
"label": "Type",
"value": [
{
"label": "Type",
"value": "Still Image",
"attrs": {
"lang": "en",
"ns": "http:\/\/purl.org\/dc\/terms\/type",
"classmap": "dpla:SourceResource",
"property": "dcterms:type"
},
"iri": "http:\/\/purl.org\/dc\/terms\/type",
"explain": "A Dublin Core Terms Property; The nature or genre of the resource.; Recommended best practice is to use a controlled vocabulary such as the DCMI Type Vocabulary [DCMITYPE]. To describe the file format, physical medium, or dimensions of the resource, use the Format element."
}
]
}
],
"thumbnail": "\/\/iiif.library.ubc.ca\/image\/cdm.wwposters.1-0038290.0000\/full\/80,100\/0\/default.jpg",
"attribution": "Images provided for research and reference use only. Permission to publish, copy, or otherwise use these images must be obtained from Rare Books and Special Collections: http:\/\/rbsc.library.ubc.ca",
"sequences": [
{
"@id": "\/\/iiif.library.ubc.ca\/presentation\/cdm.wwposters.1-0038290\/sequence\/normal",
"@type": "sc:Sequence",
"label": "Default",
"viewingDirection": "left-to-right",
"viewingHint": "paged",
"canvases": [
{
"@id": "\/\/iiif.library.ubc.ca\/presentation\/cdm.wwposters.1-0038290\/canvas\/p0",
"@type": "sc:Canvas",
"label": "The Kitchen is the [key] to victory. Eat less bread",
"height": 6352,
"width": 4263,
"images": [
{
"@id": "\/\/iiif.library.ubc.ca\/presentation\/cdm.wwposters.1-0038290\/annotation\/p0000",
"@type": "oa:Annotation",
"motivation": "sc:painting",
"resource": {
"@id": "\/\/iiif.library.ubc.ca\/image\/cdm.wwposters.1-0038290",
"@type": "dctypes:Image",
"format": "image\/jpeg",
"height": 6352,
"width": 4263,
"service": {
"@context": "http:\/\/iiif.io\/api\/image\/2\/context.json",
"@id": "\/\/iiif.library.ubc.ca\/image\/cdm.wwposters.1-0038290",
"@profile": "http:\/\/iiif.io\/api\/image\/2\/level2.json",
"scaleFactors": [
1,
2,
4,
8,
16,
32,
64,
128,
256,
512,
1024
]
}
},
"on": "\/\/iiif.library.ubc.ca\/cdm.wwposters.1-0038290\/canvas\/p0"
}
]
}
]
}
],
"description": "World War I poster.",
"@context": "http:\/\/iiif.io\/api\/presentation\/2\/context.json",
"@id": "https:\/\/iiif.library.ubc.ca\/presentation\/cdm.wwposters.1-0038290\/manifest",
"@type": "sc:Manifest"
}
From the JSON we can see the property 'thumbnail' which in this instance has the value:
http://iiif.library.ubc.ca/image/cdm.wwposters.1-0038290.0000/full/80,100/0/default.jpg
Now we can start to see how we could manipulate the URL to modify the image that is returned to us:
http://iiif.library.ubc.ca/image/cdm.wwposters.1-0038290.0000/[:region]/[:width],[:height]/[:rotation]/[:quality].[:format]
To find out what qualities and formats the image supports you can just remove everything after the identifier
and inspect just the image metadata for instance:
http://iiif.library.ubc.ca/image/cdm.wwposters.1-0038290.0000
For this image that would return the following JSON:
{
"@context": "http:\/\/iiif.io\/api\/image\/2\/context.json",
"@id": "http:\/\/iiif.library.ubc.ca\/image\/cdm.wwposters.1-0038290",
"protocol": "http:\/\/iiif.io\/api\/image",
"width": 4263,
"height": 6352,
"sizes": [
{
"width": 4263,
"height": 6352
},
{
"width": 81,
"height": 120
}
],
"tiles": [
{
"width": 512,
"scaleFactors": [
1,
2,
4,
8,
16,
32,
64,
128,
256,
512,
1024
]
}
],
"profile": [
"http:\/\/iiif.io\/api\/image\/2\/level2.json",
{
"formats": [
"jpg",
"png",
"gif"
],
"qualities": [
"color",
"gray",
"bitonal"
],
"supports": [
"baseUriRedirect",
"cors",
"jsonldMediaType",
"mirroring",
"regionByPct",
"regionByPx",
"rotationArbitrary",
"rotationBy90s",
"sizeByWhListed",
"sizeByForcedWh",
"sizeByH",
"sizeByPct",
"sizeByW",
"sizeByWh"
]
}
]
}
Wallpaper
Note:
If you just want a wallpaper we recommend that you use the image viewer on your item of choice to first crop the image (using the crop tool) and then click the 'image' icon to get directed to the IIIF link for the image.
From there you can then enter your screen resolution in the URL and save the image.
If you just want a wallpaper we recommend that you use the image viewer on your item of choice to first crop the image (using the crop tool) and then click the 'image' icon to get directed to the IIIF link for the image.
From there you can then enter your screen resolution in the URL and save the image.
So now that we've gone through all that work, lets request a nice sized desktop background:
http://iiif.library.ubc.ca/image/cdm.wwposters.1-0038290/full/1024,768/0/default.png
Of course since our image is sort of stretched now as it was portrait originally, we could instead request a much larger image and crop the image ourselves using some image editing software:
http://iiif.library.ubc.ca/image/cdm.wwposters.1-0038290/full/1900,/0/default.png
Or alternatively we could have just chosen an image that was already in landscape mode so this works much easier:
http://iiif.library.ubc.ca/image/cdm.tokugawa.1-0213163/full/1600,900/0/default.png
Last but not least, if we wanted to remove the excess borders we could pass through the [:region] part of the IIIF url:
http://iiif.library.ubc.ca/image/cdm.tokugawa.1-0213163/1145,374,9234,4645/1600,900/0/default.png
Tutorials
Note:
All of the tutorials below use the public Open Collections API key to make requests, you can also use the same key however by registering for your own API key will increase the amount of requests you can make per minute from 10 to 200
All of the tutorials below use the public Open Collections API key to make requests, you can also use the same key however by registering for your own API key will increase the amount of requests you can make per minute from 10 to 200
Basic search
Install Postman
If you are unfamiliar with making cURL requests, the easiest way to get started is to download the Chrome browser extension Postman. If you are using Firefox we suggest using the plugin RestClient which is similar enough for you to be able to follow along.
Request Type
The first thing you will need to do is set the 'Request Type' to a POST request as indicated in the image below:
API URL
Next you will need to fill out the API URL to https://oc-index.library.ubc.ca /search?api_key=ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c as shown in the following image:
Body Type
Next you are going to want to click the 'Body' tab and set the 'Body' to 'raw'
Request Body
Lastly, copy the following JSON into the 'Body' field and press 'Send'
Need help building a query?
Our Research Page has a very handy GUI tool for creating search queries to give to the API.
Our Research Page has a very handy GUI tool for creating search queries to give to the API.
Result
After pressing 'Send' you will see the ten results of your search as seen below:
Search by field
To search by a specific field you need to switch from using the query string syntax to the query match syntax. We also can ask the API to only return specific fields that we request, allowing for more useful results as shown below.
Request Body
Instructions
As before setup a POST request in Postman using the above request body as 'Body Raw'
Response
Scripts
Scripts for working with the API can be viewed or downloaded below.
Get all items from a collection
To get all items from a collection you need to request the collection first and retrieve all its item ids and then loop through each item using the API.
// Author: Yves Beaudoin
package main
import (
"encoding/json"
"log"
"fmt"
"github.com/christophwitzko/go-curl"
"strings"
)
const (
// Replace this public key with your API Key
_apiKey = "ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c"
// Define the collection to get all items from
_collection = "darwin"
)
// Set up the data types for unmarshalling the JSON search results
type _searchResult4Collection struct {
DATA []struct {
ID string `json:"_id"`
} `json:"data"`
}
type _searchResult4ItemData struct {
DATA interface{} `json:"data"`
}
func main() {
var (
collectionItems _searchResult4Collection
err error
itemData _searchResult4ItemData
itemBytes []byte
items [][]byte
)
// Get all the item ids for the collection
curlOutput := curlGet("https://oc-index.library.ubc.ca/collections/" + _collection +
"/items" +
"?api_key=" + _apiKey)
// Store them in a slice we can use
if err = json.Unmarshal(curlOutput, &collectionItems); err != nil {
log.Fatalln("\a[json.Unmarshal] ", err)
}
// Loop through each item id and store the associated JSON data
for itemIdx, item := range collectionItems.DATA {
fmt.Println("Item ID =", item.ID, strings.Repeat("-", 80))
curlOutput = curlGet("https://oc-index.library.ubc.ca/collections/" + _collection +
"/items/" + item.ID +
"?api_key=" + _apiKey)
if err = json.Unmarshal(curlOutput, &itemData); err != nil {
log.Fatalln("\a[json.Unmarshal] ", err)
}
if itemBytes, err = json.MarshalIndent(itemData, "", " "); err != nil {
log.Fatalln("\a[json.MarshalIndent] ", err)
}
items = append(items, itemBytes)
fmt.Println(string(items[itemIdx]))
}
}
func curlGet(URL string) []byte {
err, curlOutput, response := curl.Bytes(URL, "method=", "GET", "disablecompression=", true)
if err != nil {
log.Fatalln("\a[curlGet] ", err)
} else if response != nil && response.StatusCode != 200 {
log.Fatalln("\a[curlGet] response.StatusCode = ", response.StatusCode)
}
return curlOutput
}
$apiURL = 'https://oc-index.library.ubc.ca/';
// Replace this with your API Key
$apiKey = 'ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c';
// Collection to get all items from
$collection = 'darwin';
$perPage = 25;
$offset = 0;
// First query the API for the count of items in the collection
$curlOutput = curlGet($apiURL.'collections/'.$collection.'?api_key='.$apiKey);
$itemCount = $curlOutput->data->items;
// Now we can work out how many pages to loop through
$pages = (int) ceil($itemCount/$perPage);
$itemIds = [];
// First we need to get all the Item Ids from the API
while($pages > 0) {
$curlOutput = curlGet('https://oc-index.library.ubc.ca/collections/'.$collection.'/items?limit='.$perPage.'&offset='.$offset.'&api_key='.$apiKey);
// Now we want to store them in an array we can use
foreach($curlOutput->data as $itemInCollection) {
$itemIds[] = $itemInCollection->_id;
}
$offset += $perPage;
$pages--;
}
$items = [];
// Next we want to loop through each item id and store the item data.
foreach($itemIds as $itemId) {
$curlOutput = curlGet('https://oc-index.library.ubc.ca/collections/'.$collection.'/items/'.$itemId.'?api_key='.$apiKey);
$items[] = $curlOutput->data;
}
// We now have our items and can use or manipulate them as needed.
echo json_encode($items);
exit;
//Simple curl function to keep code DRY, will exit on error.
function curlGet($url) {
try {
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, $url );
curl_setopt( $ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1 );
curl_setopt( $ch, CURLOPT_RETURNTRANSFER, 1 );
$curlOutput = json_decode( curl_exec( $ch ) );
curl_close( $ch );
} catch ( Exception $e ) {
var_dump($e);
exit;
}
return $curlOutput;
}
import requests, math, json
ocApiUrl = 'https://oc-index.library.ubc.ca'
apiKey = 'ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c'
collection = 'darwin'
perPage = 25
offset = 0
# Query the API for the collection item count
collectionUrl = ocApiUrl + '/collections/' + collection + '?api_key=' + apiKey
apiResponse = requests.get(collectionUrl).json()
itemCount = float(apiResponse['data']['items'])
# Figure out how many pages there are
pages = int(math.ceil(itemCount / float(perPage)))
# Loop through collection item pages to get all items
itemIds = []
for x in range(0, pages):
collectionItemsUrl = ocApiUrl + '/collections/' + collection
collectionItemsUrl += '/items?limit=' + str(perPage) + '&offset=' + str(offset) + '&api_key=' + apiKey
offset += 25
# Get list of 25 items
apiResponse = requests.get(collectionItemsUrl).json()
collectionItems = apiResponse['data']
# Add each item id to the itemIds list
for collectionItem in collectionItems:
itemIds.append(collectionItem['_id'])
# Store all the items so we can print them out later
items = []
for itemId in itemIds:
itemUrl = ocApiUrl + '/collections/' + collection + '/items/' + itemId
apiResponse = requests.get(itemUrl).json()
item = apiResponse['data']
items.append(item)
print(json.dumps(items))
Harvest full text of all items in a collection
Note: not all collections have full text, we've specifically chosen one of the smaller collections with full text to show you how to programmatically loop through items and export the full text.
// Replace this with your API Key
$apiKey = 'ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c';
// Collection we want to harvest
$collection = 'darwin';
// Setup variables
$limit = 100;
$itemIds = [];
$items = [];
// First we need to find out how many items are in the collection
$curlOutput = curlGet('https://oc-index.library.ubc.ca/collections/' . $collection.'?api_key='.$apiKey);
// Now we have the item count, figure out the page count and create an offset of 0
$itemCount = $curlOutput->data->items;
$pageCount = ceil($itemCount / $limit);
$offset = 0;
// Loop through the pages and extract the item ids into the $itemIds array.
while ($pageCount > 0) {
$curlOutput = curlGet('https://oc-index.library.ubc.ca/collections/' . $collection . '/items?api_key='.$apiKey.'&offset=' . $offset . '&limit=' . $limit);
foreach ($curlOutput->data as $item) {
$itemIds[] = $item->_id;
}
$pageCount--;
$offset = $offset + 100;
}
// Loop through the item ids and extract metadata into the $items array.
foreach ($itemIds as $itemId) {
$curlOutput = curlGet('https://oc-index.library.ubc.ca/collections/' . $collection . '/items/' . $itemId.'?api_key='.$apiKey);
$item = $curlOutput->data;
$items[] = array(
"id" => $itemId,
"title" => $item->Title[0]->value,
"description" => $item->Description[0]->value,
"fullText" => property_exists($item, 'FullText') ? $item->FullText[0]->value : null
);
}
// We now have the items stored in $items, uncomment below to check it out.
// echo json_encode($items);
// exit;
// For more fun lets add them into a CSV file. ( You could have file permission problems attempting this )
$fp = fopen($collection . '.csv', 'w');
fputcsv($fp, ['ID', 'TITLE', 'DESCRIPTION', 'FULLTEXT'], '~', '"');
foreach ($items as $item) {
fputcsv($fp, $item, '~', '"');
}
fclose($fp);
//Simple curl function to keep code DRY, will exit on error.
function curlGet($url)
{
try {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$curlOutput = json_decode(curl_exec($ch));
curl_close($ch);
} catch (Exception $e) {
var_dump($e);
exit;
}
return $curlOutput;
}
import requests, math, csv
ocApiUrl = 'https://oc-index.library.ubc.ca'
apiKey = 'ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c'
collection = 'darwin'
perPage = 25
offset = 0
# Query the API for the collection item count
collectionUrl = ocApiUrl + '/collections/' + collection + '?api_key=' + apiKey
apiResponse = requests.get(collectionUrl).json()
itemCount = float(apiResponse['data']['items'])
# Figure out how many pages there are
pages = int(math.ceil(itemCount / float(perPage)))
# Loop through collection item pages to get all items
itemIds = []
for x in range(0, pages):
collectionItemsUrl = ocApiUrl + '/collections/' + collection
collectionItemsUrl += '/items?limit=' + str(perPage) + '&offset=' + str(offset) + '&api_key=' + apiKey
offset += 25
# Get list of 25 items
apiResponse = requests.get(collectionItemsUrl).json()
collectionItems = apiResponse['data']
# Add each item id to the itemIds list
for collectionItem in collectionItems:
itemIds.append(collectionItem['_id'])
items = []
for itemId in itemIds:
itemUrl = ocApiUrl + '/collections/' + collection + '/items/' + itemId
apiResponse = requests.get(itemUrl).json()
item = apiResponse['data']
itemStore = dict()
itemStore['id'] = itemId
itemStore['title'] = item['Title'][0]['value'].encode('utf8')
itemStore['description'] = item['Description'][0]['value'].encode('utf8')
if 'FullText' in item:
# Note we are ignoring any utf8 encoding errors here
itemStore['fullText'] = item['FullText'][0]['value'].encode('utf8', errors='ignore')
else:
itemStore['fullText'] = ''
items.append(itemStore)
with open('full-text.csv', 'w+b') as csvfile:
writer = csv.writer(csvfile, delimiter='~', quotechar='|', quoting=csv.QUOTE_MINIMAL)
writer.writerow(['ID', 'Title', 'Description', 'Full Text'])
for item in items:
writer.writerow([item['id'], item['title'], item['description'], item['fullText']])
Note: to use your CSV in Excel you will need to do the following steps:
- Open Blank Workbook.
- Go to DATA tab.
- Click button From Text in the General External Data section.
- Select your CSV file.
- In Step 1 of the wizard set Original Data Type to 'Delimited' also check the box My data has headers
- In Step 2 of the wizard in the Delimiters section, un-check the 'Tab' option, then check the option 'Other' and insert the value ' ~ '.
- Press Finish
Download Collection Data
This script should be run from a terminal window, it allows you to download items metadata into a format of your choosing using the API.
// Replace this with your API Key
$apiKey = 'ac40e6c2cb345593ed1691e0a8b601bba398e42d85f81f893c5ab709cec63c6c';
// Replace this with where you want the downloads to go
$dir = '/ocdata/downloads';
ini_set('display_errors', 1);
ini_set('log_errors', 1);
ini_set('error_log', '/ocdata/oc-downloader.log');
function displayHelpMessage()
{
echo "_________________________________________________________________\n";
echo " OC DOWNLOADER HELP\n";
echo "\n";
echo " --cid collection to ingest from\n";
echo " --fmt output format\n";
echo " json json - ubc metadata manual format\n";
echo " ubc+json json - with keys are ubc property tag\n";
echo " ld json-ld- keys are iri\n";
echo " ld+json json-ld- keys are iri\n";
echo " ld+rdf rdf/xml (direct transform of json-ld)\n";
echo " rdf rdf/xml\n";
echo " rdf+json rdf represented in rdf+json\n";
echo " turtle rdf (turtle))\n";
echo " ntriples rdf (ntriples)\n";
echo " --help show this message\n";
exit;
}
$flags = [
'--cid' => FALSE, // collection to ingest from
'--fmt' => 'rdf' // output format
];
array_shift($argv);
while ($arg = array_shift($argv)) {
switch ($arg) {
case '--help':
displayHelpMessage();
exit;
case '--env':
case '--cid':
case '--fmt':
$flags[$arg] = array_shift($argv);
break;
case '--txt':
$flags[$arg] = 'true' === array_shift($argv);
break;
default:
echo "Found [{$arg}] - no processing command is stated for this argument";
}
}
if (!$flags['--cid']) {
echo " Error: You must specify a collection [--cid] to process, see --help for more\n";
exit;
}
$extension = [
'json' => 'json',
'ubc+json' => 'json',
'ld' => 'json',
'ld+json' => 'json',
'rdf+json' => 'json',
'ld+rdf' => 'xml',
'rdf' => 'xml',
'turtle' => 'txt',
'ntriples' => 'txt'
];
$format = $flags['--fmt'];
$ext = $extension[$format];
$ocREST = 'https://oc-index.library.ubc.ca';
// Create directory if doesn't exist
if (!is_dir($dir)) {
mkdir($dir, 0777, TRUE);
}
// Get list of all collections if --cid all has been passed
if ('_all' === "{$flags['--cid']}") {
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, "{$ocREST}/collections?api_key={$apiKey}");
$response = curl_exec($ch);
curl_close($ch);
$collections = json_decode($response, TRUE)['data'];
} else {
$collections [] = "{$flags['--cid']}";
}
// Process each collection
foreach ($collections as $idx) {
// Create a directory for each collection
$dlddir = $dir."/{$idx}";
if (!is_dir($dlddir)) {
mkdir($dlddir, 0777, TRUE);
}
echo "Generating Download Files: {$idx}\n";
$finishedProcessing = FALSE;
$limit = 100;
$offset = 0;
while (!$finishedProcessing) {
echo("GET:/collections/{$idx}/items?api_key={$apiKey}&limit={$limit}&offset={$offset}\n");
// Get items from collection
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, "{$ocREST}/collections/{$idx}/items?api_key={$apiKey}&limit={$limit}&offset={$offset}");
$response = curl_exec($ch);
curl_close($ch);
$items = json_decode($response, TRUE);
$items = $items['data'];
if (empty($items)) {
$finishedProcessing = TRUE;
continue;
} else {
foreach ($items as $item) {
$iid = $item ['_id'];
echo(" - GET:/collections/{$idx}/items/{$iid}/output-format/{$format}\n");
// Get item data in format requested
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, "{$ocREST}/collections/{$idx}/items/{$iid}/output-format/{$format}?api_key={$apiKey}");
$response = curl_exec($ch);
curl_close($ch);
switch ($ext) {
case 'json':
$d = $response;
$d = json_decode($d, TRUE);
$d = json_encode($d);
file_put_contents("{$dlddir}/{$iid}_{$format}.{$ext}", $d);
break;
default:
file_put_contents("{$dlddir}/{$iid}_{$format}.{$ext}", $response);
}
}
}
$offset += 100;
}
echo "Finished Generating Download Files: {$idx}\n";
}
echo "Finished Generating Download Files. Goodbye!\n";
exit;
Sooon.