1

I am working with a shapefile in ArcGIS Pro that has 3.2 million observations and about 20 columns. The .shp file is 87MB and the .dbf is just over 2GB. I am attempting to accomplish an export.

When I export the data via the standard ArcGIS Pro instructions (http://pro.arcgis.com/en/pro-app/help/data/tables/export-tables.htm), the table successfully exports as a .csv but at around row 1.6 million all the observations begin to repeat the same observation. No observations are dropped or added, and the original shapefile remains intact.

Has anyone else experienced this?

Edit: after replicating the issue in QGIS, it seems that it has trouble getting past the 1.6millionth observation, and then copies that same observation down. In QGIS, it simply halts at that 1.6millionth observation (i.e. feature count directly differs between shapefile and the resulting table).

  • The .shp file is 87MB and the .dbf is just over 2GB. – Reputable Misnomer Jan 21 at 21:54
  • 4
    From the documentation - "There is a 2 GB size limit for any shapefile component file..." – user2856 Jan 21 at 22:07
  • Ah - classic, should have read the documentation. I will take steps to break up and iterate the process. Thanks! – Reputable Misnomer Jan 21 at 22:23
  • @Luke do you have any suggestions as to how I can work around this issue? Have tried several solutions to no avail... – Reputable Misnomer Jan 21 at 22:33
  • 2
    Your shapefile may be corrupt. I f you can recreate it using a file geodatabase, you'll be able to store much larger datasets. Try exporting to a FGDB feature class first. – user2856 Jan 21 at 22:39
1

From @Luke in the comments - there is a 2 GB size limit for any shapefile component file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.