Since building these applications I’ve been wondering, do certain topics (or tags) also relate to a colour? Does a search for Paris return the colourful images your imagination expects? Are images tagged with red really red?

With a bit of help from the Flickr API, I’ve built an application that queries the 50 most interesting Flickr Commons images for a particular tag, and displays the colours of these images. It also attempts to create a definitive colour for the tag by averaging the colours out.

As you explore the tags more & more you tend to find that most tags return an average muddy brown colour. I suspect this is partly to do with many of the images being black & white & skewing the process.

It’s really interesting to explore a few different subjects and seeing what results appear.

Formats

Can we find an colour gamut for a format?

Cities and countries

Do different cities or countries have different colours associated with them?

Objects

Do objects have particular colours associated with them? Take a bridge. Why do bridges exist? They exist to allow us to go over a river or a valley. With that logic we should expect photos tagged with bridge to have a reasonably large amount of green or blue in the image.

Sure enough, we get quite a few images with green and blue in them.

Colours

Of course colours are a natural subject to test.

Have a go

Feel free to explore the application and find some interesting results. The URL is totally hackable if the tag you want to test isn’t part of the initial tag cloud.

One thing I’ve noticed about reading the blog posts is how similar they are to Twitter posts. Many of the entries are very short and the manner in which they are written is typical of what you would find in a tweet. I went back through the diary entries to analyse their content and measure the number of characters in each entry. The average number of characters per diary entry between 2nd Feb 1915 and 21st April 1915 was 342 characters. The longest diary entry so far has been 4066 characters long, but many of the entries are less than 250 characters, and really are just short snippets of information about the events of the day. They aren’t beautifully written entries.

Reading through the diary I’m convinced that if Herbert Vincent Reynolds had access to Twitter back in 1915, he would have used it to post his diary entries. The similarities in the writing styles and structure in the methods of communication nearly a hundred years apart is uncanny. It’s also interesting to note that the Australian War Memorial is using their Twitter feed to promote the diaries.

For other collection formats like images or audio recordings, this traditional metadata is still the main method of discovering items. What I wanted to look at was the concept of a full text search for images. To do this I carried out some experiments in facial recognition and colour analysis over the photographic collection of the National Library of Australia.

Here are the slides of my presentation and a link to the search by colour application I developed as part of my research.

YQL is a service from Yahoo that provides a SQL like environment for querying, filtering and joining web services. So instead of having to write a complex URL to access data from a website, we can use YQL to write a statement that is similar to an SQL query that we might use to obtain data from a MySQL database, except, instead of querying a database, we are querying a web service. As an example, you can enter the following into the YQL console to extract photos of the Sydney Harbour Bridge from Flickr:

SELECT * FROM flickr.photos.search WHERE text="sydney harbour bridge";

When YQL was launched it initially had options to query only Yahoo’s services. If you wanted to query a web service that was outside of Yahoo’s services you were out of luck. Since then Yahoo has allowed developers to build YQL Open Data Tables. An Open Data Table is an XML file that acts as a bridge between your API the YQL language and you describe how your API is structured in terms that YQL can understand.

If we wish to use an API to return data from one of the Library’s services, say Picture Australia, we can query it using the following URL:

http://librariesaustralia.nla.gov.au/apps/kss?action=OpenSearch&targetid=pictaust&searchTerms=Sydney+Harbour+Bridge&startPage=1

As you can see, it starts to become a fairly complex URL with a lot of querystring values to point towards where we need to extract the data from.

Now let’s create that same query using YQL. Firstly I created an Open Data Table for Picture Australia. This is the key component that ties Picture Australia and YQL together. If you now enter the following into the YQL console & you’ll get back an XML feed from Picture Australia for the pictures of the Sydney Harbour Bridge.

USE "http://www.paulhagon.com/yql/pictureaustralia.xml" AS pictureaustralia;
SELECT * FROM pictureaustralia WHERE searchTerms="sydney harbour bridge" AND startPage="1";

Alternatively you can query The National Library of Australia’s catalogue for pictures of the Sydney Harbour Bridge by using this Open Data Table and entering the following term into the YQL console:

USE "http://www.paulhagon.com/yql/nla.xml" AS nla;
SELECT * FROM nla WHERE lookfor="sydney harbour bridge {format:Online AND format:Picture}";

So how is this interesting? Can’t all of this information already be gathered from our standard API’s? There are a couple of advantages to using YQL. One advantage is being able to extract just portions of the data. Say you want to extract just the title, description and persistant URL of the records and you only want to return the first 3 items, you can just enter:

USE "http://www.paulhagon.com/yql/pictureaustralia.xml" AS pictureaustralia;
SELECT title,description,link FROM pictureaustralia WHERE searchTerms="sydney harbour bridge" AND startPage="1" LIMIT 3;

or you could just extract a link to where the most relevant original item is stored.

USE "http://www.paulhagon.com/yql/pictureaustralia.xml" AS pictureaustralia;
SELECT enclosure.url FROM pictureaustralia WHERE searchTerms="sydney harbour bridge" AND startPage="1" LIMIT 1;

This starts to give you a bit of flexibility in the fields and amount of data that is returned and limit the amount of parsing that you have to do. All the hard work is being done by the servers at Yahoo.

But the really fun stuff starts when you try to create a little mashup by combining data from different services. Let’s use YQL to find the current number 1 artist at Yahoo’s music service:

SELECT name FROM music.artist.popular LIMIT 1;

We can now easily combine this search with a search for the top 5 items from or about that artist in the National Library’s catalogue:

USE "http://www.paulhagon.com/yql/nla.xml" AS nla;
SELECT * FROM nla WHERE lookfor IN (SELECT name FROM music.artist.popular LIMIT 1) LIMIT 5;

Once we have constructed this query, we can access that using a JSON-P call and use a little bit of JavaScript to display the results within a web page (see example 1).

<div id="nla"></div>
<script type="text/javascript">
function nlabooks(o){
  var f = document.getElementById('nla');
  var out = '<ul>';
  var books = o.query.results.item;
  for(var i=0,j=books.length;i<j;i++){
    var cur = books[i];
    out += '<li><a href="' + cur.link + '">'+ cur.title +'</a></li>';
  }
  out += '</ul>';
  f.innerHTML = out;
}
</script>
<script type="text/javascript" src="http://query.yahooapis.com/v1/public/yql?q=USE%20%22http%3A%2F%2Fwww.paulhagon.com%2Fyql%2Fnla.xml%22%20AS%20nla%3B%0ASELECT%20*%20FROM%20nla%20WHERE%20lookfor%20IN%20(SELECT%20name%20FROM%20music.artist.popular%20LIMIT%201)%20limit%205%3B&format=json&diagnostics=false&callback=nlabooks"></script>

We’ve now got a little widget that we can use inside any page to dynamically mashup 2 separate data sources.

If we were to do that in a traditional manner we would have to be writing two separate calls to the web services and possibly parsing the results in different ways. By using YQL, all that hard work can be carried out in a minimal amount of code.

Building these tables was as much a case of learning a bit more about YQL and the possibilities that it can offer. What I’ve shown here is a simple demonstration at the ease with which you can use services like YQL to expand your data to a wider audience.

Note: Please don’t build any mission critical applications using these data tables – they are only there for demonstration purposes. I’ll hopefully make them more permanent and hosted on the National Library’s servers.

Typically, a large dataset has a search box as it’s main interface. I wanted to get right away from that approach and create an engaging interface. This uses a map interface to allow the user to freely explore the content.

It currently uses a combination of API’s from Google and Flickr to convert a latitude and longitude from the map to obtain a place name. It then displays a shapefile from Flickr to approximate the area being searched, and returns a list of relevant results from DigitalNZ. Since I started work on this, the data returned from both of these API’s have been released under a Creative Commons license (Yahoo have released their geoplanet data and Flickr have release their shapefile data). I’ll end up incorporating these releases into the application rather than relying on the API’s for the functionality.

Explore the contents of DigitalNZ.

there seem to be more people using twitter apps than twitter web. What is twitter doing wrong?
@katykat

In April 2008, ReadWriteWeb carried out a study How We Tweet: The Definitive List of the Top Twitter Clients that showed that only 56% of Twitter users used the web interface. My gut feeling tells me that that figure is lower now, given the growth of use of devices like the iPhone. 

This is a perfectly valid question to be asking in the context of a traditional website. But Twitter isn’t a website, it’s more than that, it’s a service like email. You are not restricted to interacting with your email via one particular method. Likewise, by building upon Twitter using their API’s you are not restricted to using their service in the one and only way that you can, you have choice in how you interact with their service. The important thing isn’t the website, it is the service. Twitter.com could basically become a one page website and as long as the API’s were maintained the service would continue as normal for much of the twitter community. The user has a variety of choices in interacting with the service based upon their personal preferences. They can choose the relevant application based upon interface they like and the features they are going to use.

Flickr, despite having a far greater number of API’s available, hasn’t followed the same path as Twitter. Most people still interact with Flickr via the standard web interface. This is mostly due to their terms of use which forbids people replicating the user experience of Flickr:

Use Flickr APIs for any application that replicates or attempts to replace the essential user experience of Flickr.com

Rev Dan Catt who up until recently worked at Flickr said:

I’ve often joked that I could probably get more stuff done working with the Flickr API outside of Flickr than inside.

 So to answer the question, I really don’t think Twitter is doing anything wrong, they are doing everything right.

What can Libraries learn from what Twitter and to a lesser degree Flickr, are doing? Can we start to think about our catalogue (or other core services) not as a website, but as a service. The website version of the catalogue may just be one aspect of the delivery mechanism for the information we wish to distribute. Why can’t we look at providing our services to our users in any way they wish to be able to interact with them?

Why can’t we provide specialised access to our catalogues to specific user groups, so they (or anyone) can create:

• a simplified interface for high school users without all the complex features they don’t use
• allow an historical society to create an application based upon their needs
• an complex view of the catalogue for academics or librarians
• a visual or geographic based search
• a social network based around the catalogue

Institutions like the Brooklyn Museum and collaborative efforts such as DigitalNZ are providing their content to developers to do exactly this sort of thing. It’s very early days still and it will be interesting to see what starts to develop.

Let’s start thinking about interacting with the service, not the website.

The google maps API has geocoding API call that translates a human readable address into a latitude and longitude.  So if we pass the title of a photo into the API – let’s say “Willow Street, No. 113, Brooklyn”, it returns the latitude and longitude of “40.6978614, -73.9955804″.

For the demonstration I’m using a KML file.  Generating this file is now a 2 step process, import the data from Flickr using their API, pass the title of the photo into the Google Maps API to get the latitude and longitude and merge both results into a KML file.

Of course some of the titles provide ambiguous addresses or don’t provide enough information and don’t automatically return a result.  for some of the images I’ve manually tweaked the data that I’ve passed into the geocoding API to obtain a result.  The results are by no means perfect, but it’s a pretty good demonstration of what can be achieved from very little data and automating everything.

Please explore my New York then and now mashup and let me know what you think.

For example – thanks to free wifi from the Apple Store in Sydney on George Street, I built a web application to show me my current location.  It’s pretty accurate, but not perfect.  In this case it’s showing my location as about 100m from where I actually am (on the corner of King St and George St).  I can then display all the images from Flickr Commons that relate to that area.  As you can see there are a variety of historic images available and I’ve selected an image showing Martin Place.

From where I am standing on George Street I can see the real life buildings that are shown in the historic photo.

If I then walk down George Street 100 metres I can be in the same environment where the photos were taken and can compare the historic image of my current location on my laptop to the environment I am standing in.

So how does this work?

Up until a few months ago, the only option available was to to guess the location of the user based upon their IP address.  This might have been able to give the application the city that a user was in, but it was unlikely to provide it with a more accurate location than that.  Recently two plugins based around the W3C Geolocation specification have been developed - Geode from Mozilla labs for Firefox 3 and geolocation functions have been added to Google Gears which is available for a variety of browsers.  By using these plugins I can determine a reasonably accurate latitude and longitude for a users location and if a user doesn’t have either of these plugins installed, or for privacy reasons decides not to allow these to broadcast their locations, I can fall back to using the much less accurate IP address lookup.

Once I have a users location, I can use the Flickr API to return all the images that are within a certain radius of the user.  If I also use Google Maps as the mapping application, I can also add wikipedia articles from the users current location into the mix.

Try it out

I’ve developed two prototypes, the first one returns historical images from Flickr Commons. Given that there aren’t a vast number of photos in the Commons yet, and even fewer have geolocation information added, unless you are around the Sydney CBD area, your mileage may vary.

The second prototype returns images from Flickr that have a creative commons license. As there are a lot more images available in this category, the chances of getting a result is much greater.  For either of these to work you’ll need to download one of the plugins.

• images from Flickr Commons in my location
• images from Flickr in my location