1

I have a .gejson file that's about 378 meg (378,776k) and I'm trying to convert it to topojson.

I was successful with files that sized to about 262 meg but files greater than that failed leaving the error:

Error: "toString()" failed
at Buffer.toString (buff.js:480:11)
etc....

I'm using the command line format: ex)

mapshaper - i bigfile.geojson -o little.json format=topojson 

and I'm starting to think that this is a buffering issue for memory that needs to be changed/tweeked.

What exactly do I need for files >~262 meg?

1

Yes, this is related to the file size. It seems that mapshaper hits node's default memory limit if the file is too big.

From the Node FAQ:

by default v8 has a memory limit of 512MB on 32-bit systems, and 1.4GB on 64-bit systems. The limit can be raised by setting --max_old_space_size to a maximum of ~1024 (~1 GB) (32-bit) and ~4096 (~4GB) (64-bit), but it is recommended that you split your single process into several workers if you are hitting memory limits.

Assuming that you're on Linux or OS X, you may try to increase Node's memory hard limit with a command like this:

node --max_old_space_size=4096 $(which mapshaper) - i bigfile.geojson -o little.json format=topojson

Also, I haven't tried it, but I've heard that Firefox can handle large strings as well. So an alternative solution could be using Mapshaper GUI on Firefox.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.