A man walks into a reading room, hands over his iPad and says “Hi, my name is Jake and I’ve built this”…

Let’s go back in time

In March 2011 the National Library placed a dataset of our sheet music collection up at data.gov.au. This was to be used as one of the datasets for the LibraryHack competition. The dataset contained around 11,000 items from our sheet music collection, all the metadata, all the details about every page within each piece of sheet music and most importantly references to each image of the digitised page. This was a bit of a handcrafted dataset. Our regular API’s, like the Trove API, will only return information for the “top level” of the item, not the lower level details of every page.

We didn’t quite know what would happen with the dataset & how it would end up being used in the competition. Unfortunately it didn’t really get used in a major way in any of the entries and the dataset sat there gathering electronic dust.

Nearly 12 months later, totally out of the blue, someone walked into the reading rooms at the library, approached the person on the desk (who, as luck would have it was Sarah who was the project manager for our iPhone catalogue app), introduced themselves and said “Hi, my name is Jake & I’ve built this” & showed off what was the first prototype of what went on to become Forte.

At the time Jake was looking for a large dataset to help him solve a problem he was working on. He found our sheet music collection & built a proof of concept that helped him to solve his problem. Jake spent a lot of time at the Library working (and using our free wi-fi) and never realised that we had such an interesting collection. In approaching the library he wanted to find out what we thought. He had taken it as far as he could, but wanted some advice on how to take it further.

Over time, the Library worked with Jake to fine tune the app. The app initially started as an A-Z list of 11,000 items. We broke it up into decade by decade & attempted to show how many items were contained within each decade.  We also added a feature to be able to limit it to display who were the active composers within each time period. One thing we decided quite early on in the discussions was that there was to be no search. This was to be a discovery tool that displayed everything. We wanted people to explore.

The final app really has kept the core of Jake’s initial idea. It was refreshing to see someone outside the Library who was unfamiliar with the collection, navigate it & pull out the meaning of what was important and didn’t have preconceived ideas of how a collection had to be accessed. Likewise it was great being able to work with Jake & get the benefits of how he had approached similar problems within other non-library projects.

In keeping with the initial goals of making the data openly available, the Library has released the source code to the app under an open source license. Others can build upon what we have done.

Lessons

To me there were three really important lessons to come out of this:

1. Make your data available.
2. Don’t expect things to happen immediately
3. If you can, guide the developer to help them fine tune their product & give them insight into the intricacies of the data.

It was one of those perfect examples of everything falling into place with the right people being in the right place at the right time.

Forte is available now at the AppStore. I’m really excited about the story of how the app came into existence, our team that worked alongside Jake to take his idea & build something that I’m really proud of.  I hope you enjoy discovering our collections.

Today Flickr Commons turned 5 years old. The Commons has turned into an incredible resource of over 250,000 images from 56 different libraries, archives, and museums throughout the world. For me, the launch of Flickr Commons heralded what turned out to be a huge turning point for my career.

Flickr Commons allowed me to build my first proper mashup – my Google Streetview Then & Now. This was a bit of a breakthrough moment for me. It was my first demonstration of the power that comes from having interesting photos and associated data that was freely licensed and freely available to be shared through an API so others could do things with it. It made my day when George Oates at Flickr saw it and called me a superstar!

A bit to my surprise, people liked what they saw. I overcame my fear of public speaking and starting talking about what I had done and what could come from sharing and reusing data at conferences. People outside of work took an interest in what I had to say. People were willing to fly me places to listen to what I had to say & to listen to my ideas. Not only that, people referred to what I was doing in their conference papers and blogs. This led to me becoming the first Australian to be named as a Mover & Shaker of the library world by Library Journal. It’s allowed me to become a bit of an experimenter at work and

Along the way, my Commons experiences have introduced me to so many like minded people throughout the world. It’s these connections that can’t be measured by the number of views or comments an image contains.

In 2011 my Commons experience was complete when my work set up our own Commons account. I’m now on the other side trying to get interesting things from our collections out there to see what other clever people do with it. It’s a blast!

Little did I know that way back then that building a mashup at 3am one morning would influence my life and my career. I’m absolutely loving the journey it’s taking me on and can’t wait to see what happens next.

Thanks Flickr for what you’ve created.

In the past I’ve spoken about moving our collections Beyond the search box, the colorful library and that libraries should be the provider of digital information but not control how we interact with that information.

Recently the National Library of Australia released digitized versions of the Australian Women’s Weekly. The existing way of accessing this collection is via a search box or a calendar. That’s a pretty traditional method for accessing library information, but I want to relate to the material in a different way. Back in the physical world, how do we view magazines in a newsagents? Are magazines hidden away or are the covers displayed to catch our interest and therefore purchase them? Some magazines like National Geographic are easily recognised by their iconic yellow and striking cover images.

Likewise, the covers of the Women’s Weekly are an iconic historical record of Australian society. I can remember what the covers looked like from when I was growing up, but I can’t easily remember the dates of any of the issues. How can I access this information in a visual manner?

With a bit of screen scraping I can build an alternative entry path into the collection. By extracting the relevant details for the year, month & issue I can repurpose the data into a visual timeline.

50 years worth of issues is a bit much to be loading all at once, so I’ve built this to dynamically load in another year of covers as you scroll to the bottom of the screen. I like this interactivity as it encourages exploration without being too resource intensive. Of course the covers link to the relevant issue within Trove where you can explore the content further.

I hope this proves to be an interesting way of interacting with the collection. Enjoy.

Update 24 Dec 2010: My timeline has been integrated into Trove.

I’ve been expanding upon the experiments I presented at VALA earlier this year where I built a search by colour application for the National Library of Australia. Out of curiosity I built the same search by colour application using approximately 35,000 images from Flickr Commons.

Since building these applications I’ve been wondering, do certain topics (or tags) also relate to a colour? Does a search for Paris return the colourful images your imagination expects? Are images tagged with red really red?

With a bit of help from the Flickr API, I’ve built an application that queries the 50 most interesting Flickr Commons images for a particular tag, and displays the colours of these images. It also attempts to create a definitive colour for the tag by averaging the colours out.

As you explore the tags more & more you tend to find that most tags return an average muddy brown colour. I suspect this is partly to do with many of the images being black & white & skewing the process.

It’s really interesting to explore a few different subjects and seeing what results appear.

Formats

Can we find an colour gamut for a format?

Cities and countries

Do different cities or countries have different colours associated with them?

Objects

Do objects have particular colours associated with them? Take a bridge. Why do bridges exist? They exist to allow us to go over a river or a valley. With that logic we should expect photos tagged with bridge to have a reasonably large amount of green or blue in the image.

Sure enough, we get quite a few images with green and blue in them.

Colours

Of course colours are a natural subject to test.

Have a go

Feel free to explore the application and find some interesting results. The URL is totally hackable if the tag you want to test isn’t part of the initial tag cloud.

For the past few months I’ve been following a blog set up by the Australian War Memorial where they are recreating the diary of Herbert Vincent Reynolds by posting the entires from his diary on the days they were written. Herbert Vincent Reynolds enlisted in the First World War with the 4th Field Ambulance and went on to serve at Gallipoli.

One thing I’ve noticed about reading the blog posts is how similar they are to Twitter posts. Many of the entries are very short and the manner in which they are written is typical of what you would find in a tweet. I went back through the diary entries to analyse their content and measure the number of characters in each entry. The average number of characters per diary entry between 2nd Feb 1915 and 21st April 1915 was 342 characters. The longest diary entry so far has been 4066 characters long, but many of the entries are less than 250 characters, and really are just short snippets of information about the events of the day. They aren’t beautifully written entries.

Reading through the diary I’m convinced that if Herbert Vincent Reynolds had access to Twitter back in 1915, he would have used it to post his diary entries. The similarities in the writing styles and structure in the methods of communication nearly a hundred years apart is uncanny. It’s also interesting to note that the Australian War Memorial is using their Twitter feed to promote the diaries.