2

I am trying to calculate the coverage that a set of points has of a road network layer. I have 100 million points and approximately 80,000 segments in my road network layer.

I have created a buffer around each road segment (roadsegment.shp) and am attempting to run a spatial query (using the standard spatial query tool in QGIS) to see which segments contain at least one of my 100 million points (points.shp). However, this query is taking far too long to run and I need to find a more efficient way to do this.

I am not concerned with the number of points in each road segment buffer and I do not need to pass an attribute from the buffer to ALL of the points. I just need to know whether or not a segment contains a point. Once I know that it does, it can be dropped from the analysis. I think that this must help me to make the query more efficient.

I should point out that I am fairly new to QGIS and only know how to use the standard tools in the software.

Can anybody help me with a script to speed up the processing?

  • Removing points from the pool will NOT help you in this analysis (instead it will make the problem significantly harder). Please edit the question to specify the format(s) of the spatial data involved. – Vince Jan 17 '17 at 20:57
  • 1
    1) Don't use shapefiles. You're within 7m features of the maximum theoretical feature count (assuming you have less than 21 bytes of attributes; if you have more, you've already exceeded the maximum). – Vince Jan 17 '17 at 21:55
  • 1
    Have you created spatial indexes for all the layers in your analysis? This can make a huge speed difference in processing large datasets. – Sorbus Jan 17 '17 at 23:49
  • Hi, no I haven't. Can you tell me how to do this? – James Hobbins Jan 18 '17 at 10:04
4

Add a spatial index to each of your shapefile layers:

Layer > Properties... > General then Create spatial index.

Another way is to create them for all the layers in one go:

Vector > Data Management Tools > Create Spatial Index... then Select all then OK

1

I would suggest something like this using a virtual layer:

SELECT roads.* FROM roads, points
WHERE points._search_frame_ = roads.geometry
AND MbrWithin(points.geometry, roads.geometry)
AND ST_Within(point.geometry, roads.geometry)

where roads and points are your layers. The reason for the several lines of where clauses are to narrow it down it quickly.

Saving your layers into a SpatiaLite database would help massively.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.