Blog > 2011 > December > Screened - Every 2010 film screening in the UK on a map

Screened - Every 2010 film screening in the UK on a map

Posted: Leigh 15 Dec 2011 at 11:33 | Category: General
Tags Bfi, Data, Film, Map, Screened, Screenings, Visualisation Tweet

The BFI have been collecting data on every single film screening that's happened in the UK since 2007, and I've taken that data and put it on a map in a small web application I'm calling Screened.

At Culture Hack Day earlier this year I met Katie Ellen, then at the UK Film Council (now the BFI) and she brought along this data as something that we, the hackers and coders who'd come along, could play with. It took most of the weekend just to wrangle the data into something that could be useable in a web page - we're talking around 5.4 million screenings per year in the UK, and gigabytes of data. That's not something that counts as "big data", but nor is it "small data" when you're looking to do something at a hack day.

She spent the rest of the year getting the data cleaned and moved over to a MySQL database, which she's made available to play with online (it's _very_ slow unfortunately) and as a CSV download, table by table.

Last week over coffee she mentioned where she'd got to and I'm currently in 'hack things together over the weekend' mode so I said I'd take a look and see what I could do with the data. I often do small spare-time hack projects - mobile apps, webapps, web toys, usually around culture, social issues, democracy - the kinds of things where I can apply some of the things I've learnt to something that could have a meaningful outcome.

Introducing Screened - an interactive map of UK film screening data

(please use a modern browser - Chrome, IE9, Safari. It's not great on a phone. It's okay on an iPad).

Screened is a mapping and visualisation app for 2010 UK film data (except Northern Ireland currently). The idea was to take the large amount of film data that the BFI have been collecting and display it on a set of interactive maps on the web and see what one could find out about trends in UK film.

Essentially it answers the question "which films were screened where in 2010?"

Here's some of the stuff it does:

View a map of all screenings of all films that were publicly shown in the UK in 2010

Number of screenings: Cyan: 1-2 , Green: 10-20, Yellow: 5,000, Orange: 10,000, Red: 40,000, Magenta: 100,000.

I started with a k-means clustering display (see below), but eventually realised that what you'd want is a "heatmap" of film hotspots across the map. With helpful suggestions from Marcus Baker and David MacIver (the resident maths genii at Aframe) I managed to get it to be responsive and really quite fun to play around with.

View a map of all UK film venues that screened something in 2010

Scale of each circle indicates number of venues located near one-another.

This isn't every location that's ever shown a film (which is the data I did start with - one for the future) - instead I pulled out just the 2011 (that's the number of venues, not the near) venues that showed a film in 2010 and put them on a k-means clustering map. This means you can see a list of venues for a town with one click. Each green circle essentially collects together all venues that are geographically close so that it's easy to find them. Zooming in gives you more detail so you can distinguish between venues. (The green is incidental in this display).

View a map of specialised film screenings

"Specialised" is a term that needs some clarification - essentially it's underground, independent, unusual, art, classic, non-mainstream film and it's the kind of film that the BFI have a remit to support. I'm hoping this could be a useful visualisation for helping make decisions about where to support these kinds of film activities in any given year.

View a map of screenings for a particular film

One thing that was relatively easy to produce was a map of where a film was screened over the year. It's interesting to search for different types of films and see where in the UK that kind of film appeals. This could be very useful for festivals organisers, cinema owners, gallery curators and so on to in effect do some basic market research about which kinds of films are received well where, and where perhaps there are gaps in the market that could be served better. There's lots more that I could do here, but this is just a quick beta.

Zoom into the detail

Screened Details

"Specialised" film is shown in green, all other links are in blue.

You can get interesting ranked stats on film showings on a given area, see lists of venues, show individual films - I'm still coming up with ideas for how to drill down into the data that's most of interest.

Using the maps you can answer these questions:

"What were the top ten films that were screened in my local cinema?"
"Which parts of the UK are relatively under-served by for film screenings?"
"Which films are popular in Coventry?"
"Which venues in a town showed film in 2010?"
"Where are there hotspots of specialised film screenings?"
"Which areas of the UK are relatively under-served for specialised film?"

And a lot more...

What's next?

Cross-browser support

I'm not sure it's going to work in older browsers - I'll add something that tells you if it doesn't.

More years

The data I've based this on is just 2010. I'm hoping to extend it to every year that the BFI have data for.

An API?

I had to convert the CSV files that the BFI supplied into something that was useable and responsive - so I flipped to Mongo. It would seem like a reasonable idea to get that data onto a server that the BFI have some control over and then design a simple API for querying it. If you know what you're doing you can look under the hood - I'm making requests to a couple of 'API' type routes based on a bounding box and returning GeoJSON. I'm thinking - how about the BFI have a JSONP API that does this so that others can play with the data. I'll have a word and see if they're interested.

More maps

There are a bunch of ideas that I've had for maps. I'll make them when I have a minute.

A slider for time

I'm thinking some kind of slider at the top of each map that lets you show how the data changes over time. So for a film you could see where its first release is and then where it then goes around the country. Could be very interesting.

Other art forms?

A few people have suggested that this approach could be taken for other art forms - theatre, dance, music... if you've got a data-set that could do with this treatment drop me a line.

URLs for films and venues

At the moment it's all very javascript-heavy. I'll add Backbone.js and give these things URLs so that you can link directly to them.

The tools I used for the job

As ever, this kind of quick mashup is only possible thanks to the work of others who release open source software. Here's a bunch of stuff you should look at should you want to attempt anything similar:

Padrino - This is a great, "Rails-lite" web framework for building apps and sites. I've found that I'm using it for more or less all of my spare time hack projects, and in fact we're using it now at Aframe for many of our web-based apps.
HAML - For generating page layouts this is by far the easiest to read and debug.
Compass - A rather wonderful collection of CSS utilities and a compiler that enables you to write maintainable stylesheets that work cross-browser.
Twitter Bootstrap - Twitter have released a front-end web framework that's great for knocking something together that looks pretty and should generally work in most browsers and on many devices. iPad support is _kind_ of there, but it's not perfect. I'll be working on that.
HTML5 Boilerplate - a collection of best practices for modern web development using HTML5.
JQuery - Once your data has hit the browser, JQuery takes care a lot of the interactive elements.
Polymaps - The core of the app is the mapping display. Polymaps is designed to take a large data-set and display it incrementally on a map. As opposed to Google Maps which is designed to show an entire dataset at once. For the size of data we're talking that would make for slow requests and unresponsiveness. Additionally, the map can display extra detail at increasing zoom levels. This was essential and is also something missing from Google Maps. So much thanks to the inspiring people at Stamen who do wonderful mapping and visualisation work.
MongoDB - This is where I store all of the data for app. It's semi-relational, but also NoSQL, so you can do relational queries as well as flattening out views of data using map/reduce.
Rack/cache - I am caching each HTTP response for each call for "give me the data for everything within this bounding box" and stuffing it into...
Memcached - a very fast memory-based cache available for free on Heroku.
Varnish - HTTP-caching as much as possible for responsiveness. Unfortunately it seems that if you add Google Analytics it disables Varnish. I'm working on a fix.

For hosting, it's Heroku, with data stored with MongoHQ.

Huge thanks to Katie Ellen for giving me access to the data and for all the work getting it published. If you'd like a copy of the data yourself email her at Katie.ellen@bfi.org.uk

Got an idea?

If you've got an idea for what I could add to the app or how it could be improved, please leave a comment or tweet me: @stef

Stef Lewandowski is co-founder of Aframe. Day-to-day he's involved in software development, design and has the rather grand title of "Chief Innovation Officer".

Back To Blog