4

I'm pre-processing GPS data to assess possible methods for a planned research. The aim of the research is to use crowd-sourced cycling routes data to plan a cycling paths in an Israeli city.

Currently I'm using my own walks data to form a GIS working process. In short:

  • Data is collected using a mobile app (http://www.sports-tracker.com/dashboard), giving point location (lat,lon), altitude and a timestamp.
  • Data is parsed from its .gpx (xml) format using R. Distance in meters, speed and Z values for speed are being calculated.
  • Data is written out as polylines (for each walk); that is one line between any two points, to form one polyline shapefile for each route. The output is a .csv file with WKT as the geometry column.

Further on, I plan to overlay multiple routes from many users, to form a map of magnitude of use and reative velocities (using the Z-values). The velocities are meant to detect obstacles.

One problem is noise routes due to GPS accuracy limitations. The image below gives an example for such noise (Google maps on the background to give context):

Noise example

I'm trying to think of a way to clean the data, that is to remove the noisy paths (or nodes) an by-pass them with a straight line. The only thing that came in mind is to find self-intersections using arcmap topology; but I'm still looking for more ideas. Note that this process must be automated, since it will be preformed many times (hopefuly some thousands of times).

I'm open to qgis, grass-gis, python, arcmap and R solutions; but mainly looking for an algorithm (workflow).

Any Ideas?

Improve this question
3
  • 1
    For generality wouldn't it be better to rasterize lines and accumulate density with a cell size adequate to tell what side of the road? After all the objective is to decide where people are riding bikes and how many/often... a cold to hot raster might be the best visualization. To show the routes perhaps don't use the GPS data directly but select points from the side of the road (generated from road casings) based on nearness to the bike route... is there a particular language/API you're looking at?
    – Michael Stimson
    May 13, 2015 at 22:37
  • 2
    We have a few questions here on the subject, but none I really feel strong enough about to pick as a duplicate. You might search here on "gps clean" and "gps noise" for some ideas/information. Some of the solutions go to dedicated GPS software (such as GPSBabel)that has the ability to do analysis specific to GPS sample points. That may also cross into averaging tracks (another search term), not just cleaning them.
    – Chris W
    May 13, 2015 at 22:57
  • @MichaelMiles-Stimson I do not recject the possibility to perform a raster-based analysis and visualization yet. The only drawback here is that I will lose some attributes; and I am not sure about the number of variables I would like to work with. It is however definately an option; and if elaborated should be an answer. Regarding languages, I'm still considering, any software from the above with python or R langs might work
    – dof1985
    May 14, 2015 at 7:08

2 Answers 2

Reset to default
3

In ArcMap the Cartography tools "Smooth Line" or "Simplify Line" may give the results you want visually and can be scripted to work in model builder if you need to do the process repetitively. The down side is that smoothing the line or simplifying the line may remove some of your data for speed and barrier detection.

You could instead ignore the noise in the data collection process and "smooth" it out by having a raster function in the final analysis. I have used a 3M raster for modeling pedestrian movement in Los Angeles in the past. The raster(s) can also capture the hotspots of speed drop or increase as cell values and be reclassified and transferred back to polygons of your corridors.

Consider looking at the Portland State University "Improving Regional Travel Demand Models for Bicycling" study by Dr Jennifer Dill from about 2008. It was early GPS based bike tracking that had some of the same issues.

Improve this answer
2
  • +1 but only for the mention of the study by Dr Dill. I suspect the user doesn't want to use ArcGis (unless they really have to)..
    – Michael Stimson
    May 13, 2015 at 22:55
  • I might use ArcGIS, since it is available, but in general prefer open-source soultions. +1 For the study. @JohnT I will appreciate if you can also give a link to your work on pedestrian movements in LA.
    – dof1985
    May 14, 2015 at 7:12
2

Select single route:

enter image description here

Planarize segments, i.e. split them, where their intersection point is not the end point.enter image description here

Find shortest route between start and end point of the route: enter image description here

Delete remaining.

I've used my own scripts for this example designed to facilitate network tasks, you might consider using Network Analyst

Improve this answer
6
  • @DelixIP. Split is un-necessary since the layer is already split by nodes, i.e. it is built in that way that lines connect any two consecutive points using timestamps to get the right sampling order. Yet treating it as a network seems a good idea. My only problem here (in ArcGIS) is the utility network algorithm, which has some inconsistent. Any tools for this which are out of arc?
    – dof1985
    May 14, 2015 at 7:16
  • You must do planarise, if you want to use network. 7th node from the bottom up at the picture in the middle is a NEW node. It is the result of split of 6th and 8th original ordered segments. Without this additional node you won't get a red line, that is your corrected path. I am using networx module for this sort of games, just google it
    – FelixIP
    May 14, 2015 at 9:47
  • I see what you mean...
    – dof1985
    May 14, 2015 at 15:24
  • I like it Felix, simplified the problem nicely, but the shortest path can be a pickle all to itself.. the user prefers open source. Though shortest path is not too bad if you've only really got one choice and a few loops. Once I implemented Dijkstra's algorithm in C# over Southeast Queensland watercourses from 10K mapping.. it took over a week to complete! In the old days I would have selected where TNODE = FNODE (loop) and removed them.. now there's an idea there.
    – Michael Stimson
    May 14, 2015 at 22:49
  • Shortest path is least of my worries here, networkx module is a beauty. Planarize outside ArcGIS looks like much bigger issue to handle
    – FelixIP
    May 14, 2015 at 22:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.