1

I'm trying to map routes from an O-D survey with about 450,000 individual trips with different start and end points defined by lat, long. I want to estimate total VMT for the area by extrapolating from that. I'd been hoping to use the (really great) QNEAT3 plug-in to calculate the network cost of each route. But I've run into a problem. QNEAT3 generates a complete O-D matrix, measuring the network distance between each and every origin point and every other destination point. That's not a problem at all when dealing with a smaller data set. Since I've assigned a shared trip ID to each origin and destination pair, I'm able to just select the shortest path cost calculations I'm interested in by selecting rows where the route ID is the same for both. Then I can just ignore the "superfluous" data. The real problem is when the data set is so much bigger. Even parceling out 20,000 pairs at a time - something I can't feasibly do with so much data to get through - the number of extraneous calculations is so big that the operation inevitably fails after about 5 or 6 hours of processing.

I've spent a ton of time on this and I'm willing to learn any new skill that I might need to do this calculation. I'm just not sure what that might be. I'm very familiar with QGIS. I could probably struggle through something in R if that works. And maybe with step by step support could try to pull something off in PostgreSQL or Python. But I'm feeling pretty stuck.

Does anyone have any suggestions of how to go about this?

Improve this question

1 Answer 1

Reset to default
1

pg_routing would be a good solution. I've done several OD-matrices without any problem. To give you an idea one matrix of 16 million by 16 million took between 1,5 and two hours on a standard pc without any tuning. Take a look at this example: Creating many origin-destination routes with pgRouting

another solution might be the GRASS v.net tools (haven't tried myself) Batch distance calculation with v.net.distance

Improve this answer
2
  • Thanks, Leon. I'll give both a try. I've only tried to use PostGIS and PostGRESQL once and I didn't have particularly good luck. But I'm going to work my way through it. One concern I've got with generating the whole OD matrix is that I really need to be able to link back to the actual journeys I'm interested in to link socioeconomic data. I'll let you know how it goes.
    – Jamie DeWeese
    Commented Mar 5, 2019 at 0:35
  • That wouldn't be the hardest part of this all ;) If you have your OD pairs with trip ID's stored in a table, you can easily join the OD result back based on O and D.
    – Leonmvd
    Commented Mar 10, 2019 at 10:47

Not the answer you're looking for? Browse other questions tagged or ask your own question.