4

I am doing research in water and climate. I have downloaded a map for World Water Bodies that describes all the large water bodies that are drawn as polygons, and another map for small water bodies (World Linear Water)that are drawn as lines.

I am not a GIS expert, and I have only QGIS software, so I followed this post to load the LPK file to QGIS then I tried this post and this post to convert this layer to a point-set of Longitude and Latitude coordinates. Unfortunately, I stuck in adding the calculated fields $x and $y to the layer, as it lasts for hours without real progress but "Not responding" while the qgis-bin.exe process in the Windows Task Manger shows about 2-10GB memory and 14% CPU of a core-i7/16G-RAM PC.

Layer 1 is 528MB size, and the Nodes file is 3GB with 38 Milion features!

Is there a method to speed up the conversion?

Am I doing the correct steps?

I just want a Long/Lat list (CSV) of the points surrounding the polygons of continents, seas, lakes, and big rivers, to use them in my research.

Improve this question
3
  • 1
    With such large datasets, you could consider importing your point shapefile to PostGIS and then calculate the coordinates as described in this post: How to get coordinates from geometry in PostGIS?
    – Joseph
    Commented Apr 27, 2017 at 12:49
  • Thank you @Joseph for this advice, but I do not know what is PostGIS, is it an addon to qGIS, a standalone tool, or a python library?
    – Mohammad ElNesr
    Commented Apr 27, 2017 at 13:06
  • 1
    Sorry @Joseph, I read about PostGIS, I know now it is a database client for GIS objects, OK, I will give it a try. Thanks
    – Mohammad ElNesr
    Commented Apr 27, 2017 at 13:09

1 Answer 1

Reset to default
4

I can't test it right now but, if you want to convert the point shapefile to a CSV with lon/lat values, you may avoid using the Field Calculator (which is very time and resources consuming for 38 Million of features).

Firstly, deselect the layer from the Layers Panel and make sure that the Attribute Table for the same layer is not opened (this trick helped me in the past when dealing with some million features).

Then, right-click on the point layer in the Layers Panel, go to Save As... and use these options (you may change other options as you want, but make sure to select AS_XY in the GEOMETRY option):

enter image description here

This operation will create a CSV file which stores the same fields of the input point layer plus two additional field that store the Longitude and Latitude values. Please note that the whole operation may take some time because you are dealing with a 3GB file, but it should works (and it would be surely most efficient than using the Field Calculator in a preliminary step).

Improve this answer
14
  • Many thanks for your advice, I am trying it right now.
    – Mohammad ElNesr
    Commented Apr 27, 2017 at 12:59
  • Unfortunately, the method did not succeed! It exported just a list of shape_length and shape_area, No Latitudes and Longitudes. Example: OBJECTID,Name1,Name2,Name3,TYPE,ISO_CC,Shape_Length,Shape_Area 1, , , ,Inland intermittent,AO,0.0640761031664804,0.00016431571197887 2, , , ,Inland intermittent,AO,0.0366321064690917,3.80503219183659e-005 3, , , ,Inland intermittent,NA,0.187777497019295,0.000617306825186438 4, , , ,Inland intermittent,NA,0.0247112153483882,1.64606738052617e-005 5, , , ,Inland intermittent,NA,0.0117616719707985,5.90625617466775e-006
    – Mohammad ElNesr
    Commented Apr 27, 2017 at 13:13
  • Wait please, I might performed something wrong, I will confirm and come back shortly
    – Mohammad ElNesr
    Commented Apr 27, 2017 at 13:16
  • @MohammadElNesr This procedure must be applied on a point shapefile. Let me know if it works then.
    – mgri
    Commented Apr 27, 2017 at 13:17
  • 1
    Many thanks @mgri I applied what you told me, it took 1hr to finish, but it succeded finally. Great Answer. Here is a sample output:X,Y,OBJECTID,Name1,Name2,Name3,TYPE,ISO_CC,Shape_Leng,Shape_Area,area,perimeter 14.8248941895153,-17.1603608130011,1,,,,Inland intermittent,AO,0.064076103166480,0.000164315711979,0.000164315711984,0.064076103166480
    – Mohammad ElNesr
    Commented Apr 30, 2017 at 7:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.