3

I have a shapefile that I cannot open in any open source Python libraries, including GDAL, Pyshp, Fiona, and Geopandas. The error message: Invalid length for entity 9164 is a GDAL error message and can be found in the source code: https://fossies.org/linux/gdal/ogr/ogrsf_frmts/shape/shpopen.c at lines 871 and 2121. Both sections indicate that the error occurs during the reading/handling of offsets in the .shx file. Here is the comment to the first section that the error message occurs at: Read the .shx file to get the offsets to each record in the .shp file

The really frustrating aspect of this issue is that the file can be viewed and edited in QGIS/ArcGIS with no issues.

If I open the shapefile in QGIS and then save it with a different name, I can open the new shapefile with all python libraries. Upon further investigation, if I use the new .shx file with the original set of shp files, the file also opens in Python. So somehow this is a corrupted .shx file. Any info would be helpful as I have my processes automated.

closed as off-topic by PolyGeo Jun 3 '18 at 11:29

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This problem cannot or can no longer be reproduced. Changes to the system or to the asker's circumstances may have rendered the question obsolete, or the question does not include a procedure to enable potential answerers to reproduce the same symptoms. Such questions are off-topic as they are unlikely to help future readers, but editing them to include more details can lead to re-opening." – PolyGeo
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Corruption in shapefiles is fairly common (and fairly annoying) usually (for me) it stems from a bad download. Have a read of support.esri.com/en/technical-article/000007161 there's a utility to download that fixes some errors. – Michael Stimson May 1 '18 at 2:57
  • 2
    The odd thing here is that I thought QGIS used gdal/ogr to read shapefiles. Is it using a different version, which traps the shx corruption and falls back to the slow method of scanning the .shp? Can you edit your Q and add the version info of QGIS, and the version of GDAL that QGIS is using - you can get this from the Help -> About menu somewhere. Versions of GDAL that your python scripts use might also help, and if you can let us have your shapefile (or one that fails similarly). I might try corrupting a .shx manually and see what happens... – Spacedman May 1 '18 at 7:21
  • 2
    While it is possible to read a .shp file without .shx under most conditions, it is difficult for most clients to distinguish the difference between a "hole" in the file (where record length and offset disagree), and true corruption of the offsets file. It seems as if the code library could be less pedantic, but it's hard to blame it for trying to prevent a segmentation violation. – Vince May 1 '18 at 12:50

Browse other questions tagged or ask your own question.