Introducing the DigitalNZ Concepts API

Posted on 06 October 2015 by Thomasin

Update, November 2023: DigitalNZ no longer supports the Concepts API service. Links on this page no longer work.

Linked data experiments

At the 2013 National Digital Forum we shared our initial linked data experiments. During the intervening period we have continued to explore these ideas. Today we release the first proper iteration of our Concepts API and we are very excited! We want this data service to be simple and immediately useful. Linked data has earned a reputation of having a steep learning curve. Our aim is to create a linked data API that any interested person could pick up and start using based on a single blog post.

This is that blog post.

This initial release focuses on linking people and their works from five memory institutions. Every dot in the fuzzy image below represents a different map, photograph, book, article, audio snippet or video clip from one of these partners that we have managed to associate with a concept. Clusters of dots represent works linked with a particular person. The large clusters near the centre of the orb represent people who are associated with hundreds or even thousands of works. The stray dots around the edge are people associated with just one or two items in the partner collections. In total, this figure represents just over 4,000 people associated with about 80,000 metadata records.

A globe shape made up of small dots.

This API is tentative and we would love your help

We release this data service in a tentative state. Over the next few months I will interview developers and content partners to get feedback on the API design and the response format. We expect the API to change in response to this feedback and to become stable in 2016. We will also implement improvements to our matching algorithms and harvesting processes. Please note that the DigitalNZ non-commercial metadata Terms of Use apply to this data service.

For this initial release, we have only matched a relatively small number of partners: National Library of New Zealand, Alexander Turnbull Library, Museum of New Zealand Te Papa Tongarewa, Auckland Art Gallery and Te Ara – The Encyclopedia of New Zealand. Now that we have a pilot up, we are eager to introduce people and works from other partners. If you would like to provide API feedback or contribute metadata, please get in touch with me at develop@digitalnz.org.

While the Concepts service is in beta, its documentation lives on the DigitalNZ Supplejack pages. 

What is the Concepts API?

DigitalNZ collects metadata describing millions of images, articles, videos, datasets and audio recordings from hundreds of partner organisations. Over the years we have noticed that people frequently search for materials relating to other people. Skimming through a Google Analytics report of popular searches from the last few days, in the first few results I can see our audience seeking information on:

- “Allen Curnow” (poet and journalist).- “Samuel Marsden” (early colonial missionary)- “Edward Gibbon Wakefield” (19th C political theorist and politician)- “Ralph Hotere” (visual artist).

Many of our partners keep beautifully maintained lists of the various people who are associated with particular works. Think of an artist who created a series of paintings and sculptures, or an academic who wrote a series of articles. Although these standards are typically consistent within each institution, they tend to vary between sectors and organisations. Our Concepts work seeks to reconcile the various names of people across institutions and connect together the various items associated with them across institutions.

There are five main things that you can do with the new Concepts API.

  1. Search for a person.

  2. Get information describing a particular person.

  3. Find representations of a person across multiple institutions.

  4. See all of the records associated with a particular person.

  5. See the people associated with a particular record.

Let's go into each one of those things in a bit more depth with a mini tutorial.

A gentle introduction to using the Concepts API

1. Make sure you can view JSON

The Concepts API currently only offers responses using the JSON data standard. In order to view the data that the API sends back you will need some way to view data. I recommend you install a JSON viewer for your browser.

  • Chrome: JSON View extension

  • Firefox: JSON View extension

  • Safari: JSON formatter extension

  • Internet Explorer: I spent ten minutes looking for an easy way to view JSON in IE that ended with me slumped at my keyboard under a pile of painful work-arounds and sadness. My advice is to either switch browsers or use an online viewer like: jsoneditoronline.org .

Test your JSON viewer by visiting: api.digitalnz.org/schema. If the data is laid out in nice nested rows with colourful syntax, your JSON Viewer is correctly installed.

2. Get a DigitalNZ API key

With the exception of our the DigitalNZ Concepts Schema, all DigitalNZ API requests require an API key. If you don't already have a key, you can obtain one by creating a DigitalNZ account http://digitalnz.org/members/sign_up and visiting digitalnz.org/api_keys.

3. Search for a particular person

Please note that the following examples use a fictitious API key. When you try these out in your browser, replace the words YOUR_API_KEY with your API key.

With an API key and a way to view the data that comes back, you are ready to explore the Concepts API. First let's search for all the people named, "Colin". Copy and paste the following URL into your browser, substituting YOUR_API_KEY for your personal API key.

http://api.digitalnz.org/concepts.json?api_key=YOUR_API_KEY&text=colin

At the time of writing, you should receive a list of around 11 "Colin" results back, each containing some brief descriptive information about the person. The image below shows two of these results. It's worth spending a moment examining them. 

Code showing results for 'colin cole' and 'colin stuart lovell-smith'.

Take note of the sameAs field at the end of each Concept result. The first result is a match between an Alexander Turnbull Library authority file and a Te Papa Museum of New Zealand person page for the fashion designer, Colin Cole. (Note that we do not have a birth or death date for Colin Cole—I will return to that matter at the end of this post.) The second result is a match between the Auckland Art Gallery, Alexander Turnbull Library and Te Ara—Encyclopaedia of New Zealand for the artist and lecturer, Colin Lovell-Smith.

Just like with our Records API, the Concepts API returns paginated results

We can also search for multiple words. For example, this search will return a single result for the artist, Ralph Hotere.

http://api.digitalnz.org/concepts.json?api_key=YOUR_API_KEY&text=ralph+hotere

Code showing results for 'ralph hotere'

4. Get records relating to a specific person

DigitalNZ have assigned a numeric identifier to each Concept. These are the DigitalNZ Concept IDs. In the screenshot above, you can see that the artist Ralph Hotere has a DigitalNZ Concept ID of 448. There is a dedicated endpoint for getting information about a specific person based on their Concept ID. So, to retrieve the Ralph Hotere record, we can use the following request.

http://api.digitalnz.org/concepts/448.json?api_key=YOUR_API_KEY 

That's all well and good, but how do we find which records are associated with a person. There are a couple of ways of doing this. The simplest methods is to simply add /records to the URL immediately following the Concept ID and before the other parameters. This provides you a view of the DigitalNZ Records API, subsetted just to records associated with this concept.

http://api.digitalnz.org/concepts/448/records.json?api_key=YOUR_API_KEY

You can chain parameters to the back of this query from our Records API to alter how data gets returned. For example, we can use the &fields parameter to request only certain field values back from the API. Note in the example below how there are sometimes multiple creators associated with a work. This API offers a way to find networks of collaborators.

http://api.digitalnz.org/concepts/448/records.json?api_key=YOUR_API_KEY&fields=title,record_id,display_collection,content_partner,creator

Code showing records for Ralph Hotere.

Alternatively, you can ask for the second page of results and expand the number of results per page to 40.

http://api.digitalnz.org/concepts/448/records.json?api_key=YOUR_API_KEY&per_page=40&page=2

This query requests that no records get returned at all, instead asking for the top 50 creators, collections and content partners associated with records that were created by Ralph Hotere.

http://api.digitalnz.org/concepts/448/records.json?api_key=YOUR_API_KEY&per_page=0&facets=creator,content_partner,collection&facets_per_page=50

There are lots of possibilities here and I am excited about the things people might use this for as we expand the reach of this API. We have a second experimental method for viewing Records associated with Concepts in a true linked data view, but I will save that for another post.

Comments

Comments have been closed for this post

Really exciting work and I was all geared up to give your example query a whirl when I got the error message about my newly minted API Key being 'invalid'. Assuming I might just need to wait a day after I registered a few moments ago? Cheers Asa

--asa letourneau • 2015-10-06 00:00:00 UTC

This is nice and is a real step towards linked data. It would be great if those other APIs were linked to this one. At the moment, if you want to find a list of records related to a particular concept, you need to know about the other API, and you need to take your concept's identifier and plug it into another URI template to make another "API call". Whereas this Concept API could generate those URIs itself, and simply return them to you along with the other information about the concept.

--Conal • 2015-10-06 00:00:00 UTC

We've turned off comments here, but we'd still love to know your thoughts. Visit us on our Facebook Page @digitalnz or on Twitter @DigitalNZ to share any ideas or musings with the DigitalNZ team.