12

I'd like to map a world choropleth for display with D3, a la:

I have a dataset I'd like to display that's keyed to ISO-alpha-3 keys. So...

danger.csv
iso,level
AFG,100
ALB,0
DZA,12

etc.

Following the instructions on topojson, I know I can do...

wget "http://www.naturalearthdata.com/http//www.naturalearthdata.com/download/50m/cultural/ne_50m_admin_0_countries.zip"
unzip ne_50m_admin_0_countries.zip
ogr2ogr -f "GeoJSON" output_features.json ne_50m_admin_0_countries.shp -select iso_a3
topojson -o topo.json output_features.json --id-property iso_a3

to produce a worldmap json that is ID'd by ISO3.

My question is: at what point in the workflow should I merge the data from danger.csv onto the geo data? I had previously worked with qGIS as a GUI, but where /should/ the merge happen? In the .shp? After the ogr2ogr? Dynamically in the browser after the topojson shrink (like here http://bl.ocks.org/mbostock/4060606 http://bl.ocks.org/mbostock/3306362)?

I'm pretty good with python, but pretty new to javascript, and find myself copying and pasting Bostock examples more than actually being a generative coder there.

(I also have a related, but more involved follow-up on Stackoverflow that maybe I should migrate here: https://stackoverflow.com/questions/18604877/how-to-do-time-data-in-d3-maps)

  • I was just looking at @mbostock's examples and saw that there is one that specifically addresses GeoJoins, or "A simple script for joining a GeoJSON file with external properties in a CSV or TSV file; extracted from TopoJSON". – RyanDalton Sep 26 '13 at 3:56
11

Ask yourself two questions:

  1. Are you going to reuse the geography on multiple datasets?

    If you’ll use the same geography with multiple datasets, then it makes sense to keep the geography and data separate, and join them in the client. Many of my examples have separate CSV (or TSV) files for this reason. This way, the TopoJSON for U.S. states and counties or likewise world countries can be reused, instead of creating separate TopoJSON for every example.

    On the other hand, if you’ll only use this geography once, then you should probably “bake” the data into the geography as properties, if only to simplify the code. This approach is simpler because you only need to load a single file (so no queue.js), and since the data is stored as properties of each feature, you don’t need to join the data in the client (so no d3.map).

    Side note: TSV and CSV are often much more efficient at storing properties than GeoJSON and TopoJSON, simply because the latter must repeat property names on every object. File size can be another reason to store your data in a separate file and join it in the client.

  2. Is your data already bound to geography (e.g., a property of your shapefile)?

    Assuming you answered “no” to the first question and want to bake the data into the geography (rather than doing it in the client), how you do this depends on the format of the data.

    If your data is already a property of your shapefile, then use topojson -p to control which properties are saved to the generated TopoJSON file. You can also use this to rename properties and coerce them to numbers, as well. See Let’s Make a Map for examples.

    If your data is in a separate CSV or TSV file, then use topojson -e (in addition to -p) to specify an external properties file that can be joined to your geographic features. Cribbing the example from the wiki, if you had a TSV file like this:

    FIPS    rate
    1001    .097
    1003    .091
    1005    .134
    1007    .121
    1009    .099
    1011    .164
    1013    .167
    1015    .108
    1017    .186
    1019    .118
    1021    .099
    

    Using -e, you can map these to a numeric output property named “unemployment”:

    topojson \
      -o output.json \
      -e unemployment.tsv \
      --id-property=+FIPS \
      -p unemployment=+rate \
      -- input.shp
    

    An example of this approach is the Kentucky population choropleth, bl.ocks.org/5144735.

  • 2
    And here I was asking my hard D3 mapping questions on stackoverflow instead of gis.stackexchange because I thought there was more expertise there---and then the master himself answers my question here. =) Well, that makes 2 things I learned today. Thanks! – Mittenchops Sep 4 '13 at 21:29
3

Good question. One of the examples you provided seems to do the trick, though it is hard to follow.

You'll note that the example has two external data files, us.json and unemployment.tsv. You can think of unemployment.tsv as like your danger.csv; us.json are the geographic features with which you want to associate parameters from danger.csv. The latter, unemployment.tsv, has id and rate fields where the id is the same as the id in us.json.

It is in the client with D3 that you should merge your data and features, at least by this example. It is in the client that the unemployment rate, in this example, is joined to the county features, using the d3.map() function. This is where it is initialized:

var rateById = d3.map();

And this is where rate gets mapped on to the id:

queue()
    .defer(d3.json, "/mbostock/raw/4090846/us.json")
    .defer(d3.tsv, "unemployment.tsv", function(d) { rateById.set(d.id, +d.rate); })
    .await(ready);

I must admit I don't know what queue() is for, but it is not important to this discussion. What is important to note is that the id field in each county feature is replaced by the unemployment rate. the rate is now accessible by the shared identifier id (EDIT: As @blord-castillo points out, this is actually the generation of a new associative array, or key hash, where the rate is mapped to the id). This is where the rate is called up for the purposes of symbology (here, predefined CSS classes are available for each quantile):

...
.enter().append("path")
  .attr("class", function(d) { return quantize(rateById.get(d.id)); })
  .attr("d", path);

Where the quantize() function returns the name of the CSS class that should be used to style that feature (county) based on its unemployment rate, which is now defined in the feature's id field.

  • 1
    FYI On the queue: bsumm.net/2013/03/31/analyzing-mbostocks-queue-js.html – johanvdw Sep 4 '13 at 13:21
  • queue allows async parallel loading of the data sources instead of serial loading. – blord-castillo Sep 4 '13 at 13:25
  • 1
    What is going on in that example is that rateById is a key hash. No changes are ever made to the country features and the us.json data is untouched. Instead, unemployment.tsv is converted into a key hash called 'rateById'. rateById.set() is looped over unemployment.tsv so that a key is inserted for each id in unemployment.tsv (not in us.json) and the value of that key is set to the rate field for that id in unemployment.tsv. Later on, rateById.get() is called to use the hash to look up the unemployment rate by id; that value is used to set the style on the us.json features, then discarded. – blord-castillo Sep 4 '13 at 13:28
  • Why does this /replace/ the ID with the rate instead of attaching it as an attribute somewhere else? This would seem to make it harder to do selection later. – Mittenchops Sep 4 '13 at 13:42
  • 1
    It doesn't replace the id with the rate. It creates a lookup hash from id to rate. – blord-castillo Sep 4 '13 at 13:43
2

First off, the first row of your csv must be a comma separated list of column names to use this method. If this is not possible, add a comment about this and I will see if I can work out how to use d3.csv.parseRows instead of d3.csv.parse. d3.csv.parse is called by the assessor function on .defer(function, url, assessor).

I am going to assume your file now looks like this:

danger.csv
iso,level
AFG,100
ALB,0
DZA,12
...

Using this, you can create a lookup hash from ISO3 to danger level.

var dangerByISO3 = d3.map();
queue()
    .defer(d3.json, "url to topo.json")
    .defer(d3.csv, "url to danger.csv", function(d) {dangerByISO3.set(d.iso, +d.level);})
    .await(ready);
function ready(error, world) {
    //You now have world as your available topojson
    //And you have dangerByISO3 as your danger level hash
    //You can lookup a danger level by dangerByISO3.get(ISO3 code)
}

Code walkthrough

var dangerByISO3 = d3.map();

First you create a d3.map() object which will function as your key hash, and store this in the variable dangerByISO3.

queue()

Use queue for parallel loading.

.defer(d3.json, "url to topo.json")

Load your topojson as the first argument to be passed to the await function (after error). Note the style here where this is a chained function on queue(), but listed on a separate line (there is no terminating semicolon on queue()).

.defer(d3.csv, "url to danger.csv", function(d) {dangerByISO3.set(d.iso, +d.level);})

Two things are happening here. First, you are loading danger.csv as your second argument to be passed to the await function. As you will see below, this argument is not actually used. Instead, an assessor argument is provided to the loading function, d3.csv. This assessor will process each row of the csv. In this case, we call the set function on dangerByISO3 so that for each combination of an iso key, we set the level as the value to go with that key. The +d.level notation uses unary + to coerce the value of d.level to a number.

.await(ready);

Once both data sources are loaded, they are passed as two separate arguments to the function ready(). The first argument to the callback is always the first error that occurred. If no error occurred, then null will be passed as the first argument. The second argument is the first data source (result of the first task), and the third argument is the second data source (result of the second task).

function ready(error, world) {...}

This is the callback function ready(). First we take the error argument which should be null if the two loading tasks completed successfully (you should actually add language to catch and handle errors). Next we take the topojson data as the object countries. This data should be processed in the body of the function with something like .data(topojson.feature(world,world.objects.countries).features). Since ready() does not take a third argument, the result of the second task, our csv, is simply discarded. We only used it to build the key hash and did not need it after that.

  • Yeah, you're right, my csv does actually look like a well formed csv instead of the careless demo I posted. =) Sorry, I'll update that. – Mittenchops Sep 4 '13 at 16:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.