5

I've imported a multi-shape shapefile with ~300.000 shapes (445MB shapefile) and a CSV file with ~2.500.000 point coordinates into QGIS (100MB CSV file). I converted the CSV file to shapefile as well, resulting in a (~150MB shapefile)

My multi-shape file has a field named CD_GEOCODI with a unique code for each shape within the file. My point data file has 3 fields: lat, long and var1.

Print screen of QGIS with both: points and shapefiles

It is easy to visually spot points within the specific shapes. I would like to add the CD_GEOCODI variable from my multi-shape file to the export of the points point_data, so the resulting data will now have 4 fields: lat,long,var1,CD_GEOCODI in which the last field, CD_GEOCODI is the CD_GEOCODI variable from the shape which the point is within.

I've tried four methods:

  1. Processing > Toolbox > Saga > Vector Point Tools > Add polygon attributes to points

  2. Vector > Data Management Tools > Join by location

  3. Install Point Sampling Tool plugin > Plugins > Analyses > Point Sampling Tool

  4. Install MMQGIS plugin > MMQGIS > Combine > Spatial Join

All of them seem to work with small datasets but all of them freeze and make unpratical slow progress (based on the velocity of temp files growth in size, since QGIS is freezed) with my huge dataset.

My environment is: Win10 x64, qGIS 2.18.9, 6th generation i7-6700HQ, 8GB RAM, 1TB HD 5200

How can I achieve this goal with with those huge datasets in QGIS?

| improve this question | |
  • 2
    Possible duplicate of Assigning polygon layer attributes to point layer features in QGIS – Oto Kaláb Jun 14 '17 at 6:36
  • 1
    I edited the question with the huge files particularity – steps Jun 14 '17 at 12:12
  • 1
    Actually no. I already converted point_data to *.shp. Will edit the question to clarify – steps Jun 14 '17 at 13:04
  • 2
    With such large datasets, it might be recommended to import your shapefiles into PostGIS and use this to run your analysis. The following guide is, in my opinion, very helpful: Loading Spatial Data into PostGIS with QGIS. – Joseph Jun 14 '17 at 14:36
  • 1
    @Joseph I think your comment is worth a full answer. Maybe spatialite can cope with the huge data amount too. – AndreJ Jun 22 '17 at 5:11
1

I believe the best way to achieve that is to use: Saga-->Vector point tools-->Add polygon attributes to points. It is available through the Processing toolbox in and it gives you the option to choose which attribute you want to pass to the point layer.
Alternatively you can use Vector-->Data Management-->Join by location which creates a new point layer with all the attributes from the polygon and the points. Then you can delete the unnecessary ones.

| improve this answer | |
  • I tried both methods, they seem very good for small datasets, however they didn't work for my very huge dataset. I edited the question with that in mind. – steps Jun 14 '17 at 12:13
1

You didn't indicate the QGIS version that you are using. Strongly recommend using the latest version 2.18.9 since a major bug regarding attribute table performance was fixed in that.

I tried to repeat the same in my laptop running Win 10 x64, i7 processor (2core), 16GB ram with 5200rpm hdd.

The polygon layer is around 300MB with around 200,000 features and a point layer with 220,000 points and 800mb in size due to the number of attributes.

Using your first option; Processing > Toolbox > Saga > Vector Point Tools > Add polygon attributes to points

It took 5 min 38 seconds to complete. saga_cmd.exe used 30% CPU and 1.68GB ram.

I guess it all depends on the qgis version and hardware.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.