A simple set of scripts to help explore Tate's collection dataset.


* Licensing

Code is GPLv3+, data is CC0 with the exception of location data derived from OpenStreetMap which is ODbL. As a result the following link covers artist-locations.csv only:

http://www.openstreetmap.org/copyright


* Prerequisites

This project requires MongoDB, node.js and the following node libraries:

npm install csv
npm install mongodb
npm install node-geocoder
npm install sentiment
npm install sleep


* Importing The Tate Collection Data Into MongoDB

Check out the Tate collection git project:

https://github.com/tategallery/collection

either directly into this folder, or elsewhere and link it in as "collection":

ln -s ../../path/to/collection

Git will ignore the collection directory.

To import the records into MongoDB you have to feed records in one per line.
Like this:

find collection/artists -name *.json -exec perl -p -e 's/\n/ /' '{}' \; -exec echo \; | mongoimport --db tate --collection artists --stopOnError --drop
find collection/artworks -name *.json -exec perl -p -e 's/\n/ /' '{}' \; -exec echo \; | mongoimport --db tate --collection artworks --stopOnError --drop

Note that this will delete any existing data in these collections.


* Usage

Install the prerequisites, Import the Tate data into MongoDB, then type:

make

This will generate any csv files that don't already exist.

Building the location database takes some time and is error probe. You may need to retry several times before this completes. Progress information and errors are output to stderr. But it's worth running:

node artist-locations.js

before running make, or renaming artist-locations.js to a non-.js extension so it isn't run by make. Geolocations are stored in the database, so this is a one-time hit unless .

If you need to remake them after updating the MongoDB data, type:

make clean
make


* Naming

File names use singular or plural names to indicate whether a property can be multiple.

The "subject" entries for artworks have been named "category", "subcategory" and "subject". These are child, child/child and child/child/child in the json for an artwork respectively. Naming the most specific element of a subject, that is an entity several levels down from an array names "Subjects", as "subject" may be confusing in terms of describing the data structure but is clearer in terms of describing the conceptual hierarchy.