Automatically clean GPS based line files that have multiple traverses and GPS drift using QGIS 3

Question

Given a GPS Log with floating lines within ~5m at a time, is there any process that will automatically clean these and smooth these out?

I have tried Smooth, Simplify, Saga Line Smoothing and Cartographic Line Generalization with various options.

None of these get rid of this level of issues unless settings like the following are used

processing.run("native:smoothgeometry", {'INPUT':'Z:/My Drive/Mangoesmapping/Spatial Projects/2019/020_Adhoc_CAFNECRide/Working/2019_Route_Edit_Master/2019_CAFNEC_MGA94z55.shp|layername=2019_CAFNEC_MGA94z55','ITERATIONS':1,'OFFSET':0.5,'MAX_ANGLE':90,'OUTPUT':'TEMPORARY_OUTPUT'})

But this gets rid of the corners as well and makes the line to smooth.

Ideally I just want all the nodes between the measure tool start/end to be removed automatically and the remaining nodes to be about where the measure line starts/stops.

UPDATE 1

Based on suggestion from @babel I tried Multipart to Singlepart -- > $length --> Select < 0.5 and this does a good job of showing the problem areas.

If I now delete these I get gaps in the data and also some slightly longer slivers left

Dissolving them retains the gaps -is there a way to remove the gaps programmatically.

I've uploaded a dataset to test with at https://drive.google.com/drive/folders/1eaXpIZvUfMhzEtnGmHe9AMgH2ClQBfiX?usp=sharing

Answer 1

Interesting problem! I have written a python script which I am hoping will do what you are after. The script simplifies line features based on segment length rather than distance between vertices. I have coded a basic ui which allows for selecting the input layer and segment length (which is in the units of the input layer). When you click run, a new temporary layer called 'Simplified line' will be created with the same CRS as the input layer. For each feature in your input layer, all segments longer than the inputted value will be retained, and any gaps will be filled based on the vertices of the retained line segments. If no segments are longer than the inputted value, you will just get new, straight lines from the start and end points of the input features. All fields and attributes are copied from your input layer. I have only tested this on fairly simple data, so it would be great if you would like to try it on your dataset and let me know how it goes. Just paste into an editor in the python console and click run. If it works well, I will likely port it to a processing script.

Example:

(Input layer)

Tool ui:

Result:

class SimplifyLinesDlg(QDialog):
    def __init__(self):
        QDialog.__init__(self)
        self.set_up_ui()

    def set_up_ui(self):
        self.setGeometry(150, 150, 450, 250)
        self.setWindowTitle('Simplify lines by segment length')
        self.cb_label = QLabel('Select input layer:', self)
        self.cb_label.move(25, 30)
        self.layer_cb = QgsMapLayerComboBox(self)
        self.layer_cb.move(125, 25)
        self.layer_cb.resize(300, 20)
        self.layer_cb.setFilters(QgsMapLayerProxyModel.LineLayer)
        self.layer_cb.setShowCrs(True)
        self.label_le = QLabel('Enter segment length (layer units):', self)
        self.label_le.move(25, 105)
        self.length_le = QLineEdit(self)
        self.length_le.move(250, 100)
        self.btn_run = QPushButton('Run', self)
        self.btn_run.move(350, 200)
        self.btn_run.clicked.connect(self.run)
        self.show()

    def run(self):
        layer = self.layer_cb.currentLayer()
        src_crs = layer.crs().authid()
        if self.length_le.text()is not '':
            length_val = float(self.length_le.text())
        else:
            length_val = None
            self.msg = QMessageBox()
            self.msg.setText('Please enter a length value')
            self.msg.show()
        if length_val:
            temp_layer = QgsVectorLayer('LineString?crs={}'.format(src_crs.lower()), 'Simplified_Line', 'memory')
            temp_layer.startEditing()
            flds = [f for f in layer.fields()]
            pr = temp_layer.dataProvider()
            pr.addAttributes(flds)
            temp_layer.updateFields()
            feats = [f for f in layer.getFeatures()]
            for feat in feats:
                atts = feat.attributes()
                geom = feat.geometry()
                verts = [v for v in geom.vertices()]
                idx_1 = range(len(verts)-1)
                segment_lengths = {}
                vert_list = []
                vert_list.append(verts[0])
                for i in idx_1:
                    dist = verts[i].distance(verts[i+1])
                    if dist > length_val:
                        vert_list.append(verts[i])
                        vert_list.append(verts[i+1])
                vert_list.append(verts[-1])
                new_geom = QgsGeometry().fromPolyline(vert_list)
                new_geom.removeDuplicateNodes()
                temp_layer.startEditing()
                temp_feat = QgsFeature()
                temp_feat.setGeometry(new_geom)
                temp_feat.setAttributes(atts)
                pr.addFeature(temp_feat)
            temp_layer.commitChanges()
            QgsProject().instance().addMapLayer(temp_layer)

W = SimplifyLinesDlg()

Answer 2

This is too large for a comment so I'll put it here, but it's just two ideas you might look at. First, I just used this on some GPS tracks: Dissolve, buffer 10m with dissolve option, add field and give it a simple value like 1, rasterize (GDAL) using that value and outputting a byte raster, thin the raster (GRASS v.thin), vectorize the thin (GRASS r.to.vect) and then smooth the results using SAGA line smoothing. A simpler alternative method would be to run a clean with a large enough tolerance to snap the lines together without undue shape changes (that's always possible). I used the Integrate tool in ArcGIS because I can never get GRASS v.clean to work properly but I expect others can.

Answer 3

The process I ended up using was based on a few answers

1 - Skeletonize (multipart to singlepart / explode), Delete small features and Buffer based on Simplifying multiple lines to create central axis?

Tried -but very very slow 2 - Use the HCMGIS based on Is it possible to convert Polygon to centerline linestring? and Finding centrelines from polygons in QGIS? https://community.wolfram.com/groups/-/m/t/1533248

Instead

3 - created QGIS model that

• a - explode to vertex
• b - voronoi Poly
• c - clip by buffer
• d - poly to line
• e - explode to vertex
• f - select all vertices that intersect the lines from step 1
• g - invert selection
• h - line to point