As a followup to my previous question (Converting SimpleGeo Places GeoJSON file to shapefile), I am now struggling with what I perceive might be an ogr2ogr "memory" limitation, but I cannot be sure. I have successfully converted the SimpleGeo dump files into proper geojson files (verified that are directly readable in QGIS).

Now I want to convert this data to SHP or other formats (SQLite, PostGIS). I am running the following command with success on smaller files (where "places_fr.gejson" has a file size of ~118MB):

ogr2ogr -f "ESRI Shapefile" places_fr.shp places_fr.geojson

When I attempt to run this same exact command with a geojson file that is 340MB, ogr2ogr crashes in one of two ways:

  1. "lh_table_new: calloc failed" (first attempt)
  2. Windows throws an "ogr2ogr.exe has stopped working. Please close the program." (subsequent attempts)

I am running the ogr command via FWTools in a Windows 7 environment dual core with 8GB of installed RAM (4GB available, 12GB virtual memory available). Is there a memory limit to ogr2ogr converting these datasets? I have also tried converting into SQLite with the same results, so I don't think it is an SHP conversion issue. Thanks.

  • Did you ever get this to work? I am also now trying to convert the simpleGeo places geojson file to shp - i saw your script in the original post but I'm not sure what the infile and outfile are - in terms of the file format that can be read and written? – kflaw Mar 17 '15 at 18:32
  • nevermind i got that working now I am stuck on converting to shp – kflaw Mar 18 '15 at 16:31

Forewarning: given the amout of memory you are reporting on your system I suppose you are using a 64 bit build of Windows. If not this solution does not apply.

The memory limit for all 32 bit applications on Windows (regardless of the Windows version, so it is true also for 64 bit Windows) is 2GB. You might be hitting that limit, to confirm run ogr2ogr and check memory usage with task manager.

If the problem is indeed due to a memory issue you can do two things:

  1. build yourself a 64 bit version of GDAL (I could not find readily avail binaries)

  2. patch ogr2ogr.exe with this tool: http://ntcore.com/4gb_patch.php. After patching (make a backup copy of course) the binary should be available to address a grandtotal 4GB

  • 3
    I watched the resource monitor, and as soon as the memory reached 2GB, the ogr2ogr process crashed. It was definitely a memory limitation of 2GB within ogr2ogr/fwtools, likely because it was compiled as a 32-bit app. I patched the ogr2ogr.exe with the tool you suggested and it worked like a charm. Of course, there still is a limit where it cannot be used (like with the 7GB U.S. places files), but at least there is an explanation for why ogr2ogr is failing. Thanks! – RyanKDalton Oct 29 '11 at 16:46
  • were you ever able to process the 7GB US file? – kflaw Mar 18 '15 at 16:30

If there is a problem, it would be most likely specific to the OGR geojson driver and not OGR itself. Try using a newer binary (like the one from OSGeo4W) or even better, a 64 bit version of it. Sorry, unless you are compiling it, I do not know where you can get a 64bit binary for Windows - the Ubuntu Linux and Mac KyngChaos binaries will be 64 bit though.


May be split the file to many files and import with a batch the files.

http://www.clubic.com/telecharger-fiche184550-split-files.html or http://www.hjsplit.org/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.