6

I'm using QGIS 2.18

I have a point layer with the 4405 address positions of a municipality, as well as a road layer with road segments.

I would like to join the attribute information of the closest road segment to each point UNDER THE CONDITION that the attribute "street name" in the road layer has the same value as the attribute "street name" in the address point layer. In other words, the tool should look for the closest road segment which has the same street name and join the attribute values of this road segment to the point.

Using the NNJoin plugin in QGIS does the job for the first part of my question, but I'm not able to include my condition (as a consequence, for 302 points the attribute values of a road segment are joined for which the street name is not the same as the one indicated in the point).

Is there some kind of extended plugin of NNJoin which could do the job?

  • 1
    @Stijn Claes, why not to use as you mentioned NNJoin plugin and further proceed with "Select by expression" or some Python code? Hence in the point layer, you will have all the necessary attributes, such as "Street_Name_point", joint "Street_Name_line", and distance. The only thing you need to accomplish is to write a statement(code) which will 1) detect all records, where "Street_Name_point" == "Street_Name_line" and 2) define the min("distance") within these subgroups. – Taras Oct 16 '18 at 5:54
  • Thanks for the suggestion. I however think it will not work because the NNJoin_plugin only joins the shortest line to the address point. If I understand your suggestion correctly, you would have to have several joins between points and lines from which you select those with 1) corresponding street names and 2) shortest distance. – Stijn Claes Oct 17 '18 at 7:29
4

It just so happens that I've just made a script to do this exact operation. In the processing toolbox, click Tools > Create new script, then copy and past the below code (tested on version 2.18). Double click to run it and fill in the form. Most fields are self explanatory but the "tolerance" field specifies a distance within which to search for a segment with matching attributes.

from qgis.core import *
from PyQt4.QtCore import QVariant

##in development=group
##points=vector point
##point_attribute_to_match=field points
##roads=vector line
##road_attribute_to_match=field roads
##road_attribute_to_transfer=field roads
##tolerance=number 500

#get source and target objects
sources = processing.getObject(points)
linksLayer = processing.getObject(roads)

#add new field
sources.startEditing()
newField = QgsField('road_id', QVariant.Int)
sources.addAttribute(newField)    
sources.updateFields()
sourceIndex = sources.fieldNameIndex('road_id')

#variables for progress bar
percentMax = sources.featureCount()
p = 1

for s in sources.getFeatures():   

    #get geometry 
    if s.geometry():
        sGeometry = s.geometry().asPoint()
        if sGeometry == [0,0]:
            sGeometry = s.geometry().asMultiPoint()[0]

    else:
        progress.setText("no geometry found")

    #get nearest feature and write source attribute out
    sDistMin = 99999999999999
    sourceId = -1
    for l in linksLayer.getFeatures(QgsFeatureRequest().setFilterRect(s.geometry().buffer(tolerance,5).boundingBox())):

        print l.attribute(road_attribute_to_match)

        sDist = l.geometry().closestSegmentWithContext(sGeometry)[0]

        if sDist < sDistMin and s.attribute(point_attribute_to_match) == l.attribute(road_attribute_to_match):
                    sDistMin = sDist
                    sourceId = l.attribute(road_attribute_to_transfer)

    progress.setText("Marker")
    if sourceId != -1:    
        sources.changeAttributeValue(s.id(), sourceIndex, sourceId)
        sourceId = -1

    else:
        progress.setText("no matching source found for: " + point_attribute_to_match)

    #set progress
    progress.setPercentage(p/(float(percentMax))*100)
    p += 1   

sources.commitChanges()
  • +1. I will need to compare accuracy and performance with the second answer in order to select the bounty winner – radouxju Oct 19 '18 at 10:16
  • thank you, this is also a good answer but the double loop makes it slower than the other solution. I wish I could split the reward but this is not possible. – radouxju Oct 22 '18 at 7:05
  • @firefly-orange I am trying to replicate this code to suit points matching to points by ID and proximity. I have modified Vector Line to Vector points. The spot i am falling at i believe is the sDist = l.geometry().closestSegmentWithContext(sGeometry)[0] line. I am trying sDist = l.geometry().nearestNeighbor(sGeometry)[0] and replacing with NearestPoint to try based on the following website, link. If i use code as is, the output is 0 for my records. Any help would be appreciated. – Pat Jan 17 at 3:04
  • @pat if you post your question as a new question with a link to this one and include code and error message, I'm sure you will get the help you need – firefly-orange Jan 17 at 10:19
2
+50

QGIS 3.x

Usually I'd do this efficiently in PostgreSQL/PostGIS (and would always recommend to have a look into SQL awesomeness), but I had a simple case to solve in pyqgis a few weeks ago.

The fairly simple script evolves around creating a dict with QgsSpatialIndex objects for each distinct value in the JL_FIELD (in my case, both layers had the exact same distinct values), i.e. that attribute you want to match on, from the JOIN_LYR and execute their .nearestNeighbor() method for each point in the BASE_LYR with matching BL_FIELD.

Run this from within QGIS (Python Console) and with your layers loaded into the project:

### setup

## input vars
BASE_LYR = '<base_lyr>'     # layer to which you want to find NNs and add their attributes
JOIN_LYR = '<join_lyr>'     # layer to find the NNs from
BL_FIELD = '<bl_field>'     # field to match in BASE_LYR
JL_FIELD = '<jl_field>'     # field to match in JOIN_LYR
_PREFIX  = '<prefix>'       # prefix to add to appended fields

## working vars 
indexes = {}
attributes = {}
fields = []


### create dict of 'QgsSpatialIndex' objects for each distinct value in JL_FIELD

## process JOIN_LYR first
layer = QgsProject.instance().mapLayersByName(JOIN_LYR)[0]

## add _PREFIX to field (!not safe for provider dependent field name length!)
for field in layer.fields():
    field.setName(_PREFIX + field.name())
    fields.append(field)

## get names into list for faster lookups later
_names = [field.name() for field in fields]

## iterate features of JOIN_LYR...
for feature in layer.getFeatures():

    _mf = feature[JL_FIELD]

    ## ...and if the dict key for JL_FIELD exists, add the feature to its 'QgsSpatialIndex'...
    if _mf in indexes:
        indexes[_mf].insertFeature(feature)

    ## ...or create the key and 'QgsSpatialIndex' and insert feature
    else:        
        indexes[_mf] = QgsSpatialIndex()
        indexes[_mf].insertFeature(feature)

    ## also, extract the attributes and combine with new field names for faster lookup later
    attributes[feature.id()] = dict(zip(_names, feature.attributes()))


### get nearest neighbors and add their and attributes

## process BASE_LYR
layer = QgsProject.instance().mapLayersByName(BASE_LYR)[0]

## just to see some progress in QGIS Python Console...
_cnt = layer.featureCount()

## open layer in edit session, with auto-commit or -rollback
with edit(layer):

    ## add fields from JOIN_LYR to BASE_LYR
    for field in fields:
        layer.addAttribute(field)

    ## iterate over features
    for i, feature in enumerate(layer.getFeatures()):

        ## silly way to show percentage in 10% steps
        if i / _cnt * 100 % 10 <= 0.001:
            print('{}%'.format(int(i / _cnt * 100)))

        ## find actual nearest neighbor in 'QgsSpatialIndex' object at the dict key of this features BL_FIELD value
        nn = indexes[feature[BL_FIELD]].nearestNeighbor(feature.geometry().asPoint(), 1)[0]

        ## append the attributes
        for field, value in attributes[nn].items():
            feature.setAttribute(field, value)

        ## initialize the updates  
        layer.updateFeature(feature)

    ## done
    print('100%')

I had no time for fancyness, any form of fallback or error catching, and this has some parts rather intolerant towards uncommon values and such, so there is plenty room for conceptual improvement, but this worked for me with reasonable performance. See if you can adapt to your case...

Btw., I had about 450 distinct values (and thus QgsSpatialIndex objects) and 40k features in JL_LAYER.

Note that QgsSpatialIndex does only compare the bbox extent; for non-point geometries this will not fully guarantee the actual closest match. If you find some mismatches, you can e.g. implement a second step to measure the actual distance for when .nearestNeighbor() returns more than one feature.

  • +1. I will need to compare accuracy and performance with the second answer in order to select the bounty winner – radouxju Oct 19 '18 at 10:16
  • difficult choice, but your method is a little bit faster. You should however consider that there is not necessarily a match on, so you need to add a test before line "nn = indexes..." – radouxju Oct 22 '18 at 7:04
  • @radouxju thx. this is tailored to my usecase, e.g. the data was guaranteed to have a match for each category. I added this for the sake of concept; only a bunch of lines of code, dict driven and index powered. I also had reasonable success using multiple processes, with the memory footprint being the bottleneck. if firefly-orange's script is already suited for universal usage, and with only 'a little bit' of speed difference, his/her answer might deserve the credits? – ThingumaBob Oct 22 '18 at 7:22
  • I know, but 1) I didn't post this question, otherwise I could have accepted his answer and 2) bounty must be uniquely assigned. The most credit I could give in this case was +1. I guess that other people will find his answer usefull and also upvote. – radouxju Oct 22 '18 at 7:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.