Fastest way of adding dictionary values to attribute table with PyQGIS3

Question

I have a Python dictionary with around 450,000 keys and respective values. I want to join this dictionary to an attribute table of a vector dataset with roughly the same amount of features.

First I created a new field where the values are to be stored and I saved its index in a variable. I also saved the index of the field through which I want to join the values of the keys.

layer_prov = layer.dataProvider()
layer_prov.addAttributes([QgsField("new_field", QVariant.Double)])
layer.updateFields()
new_field_idx = layer.fields().indexFromName('new_field')
join_field_idx = layer.fields().indexFromName('join_field')

layeris the vector dataset that I want to join the information from the dictionary to. dict is the dictionary.

for key, value in dict.items():
layer.startEditing()
for feature in layer.getFeatures():
    attrs = feature.attributes()
    join_value = (attrs[join_field_idx])
    join_value = str(join_value)
    if join_value == key:
        feature[new_field_idx]=value
layer.commitChanges()

Is there any way of doing this faster? For every entry in the dictionary this model needs to go through all the features of the vector dataset (450,000 x 450,000 = 202,500,000,000‬ comparisons).

Answer 1

Instead of comparing every feature with the keys in the dictionary, I saved the whole dictionary as a csv and simply used the QGIS algorithm Join Attribute Table (probably some SQL plays its part here as well!?) to join the csv to the vector.

#create csv from dictionary
os.chdir(Output_Ordner)
header_list = ["keys", "values"]
habitat_vkp_sum_csv = open("key_value_csv.csv", "w")
writer = csv.writer(key_value_csv, delimiter=',', quotechar=',')
writer.writerow(header_list)
for key, value in dict.items():
    writer.writerow([key, value])
key_value_csv.close()

#perform table join between vector data and csv table
processing.run("native:joinattributestable", {
    'INPUT': layer,
    'FIELD': 'keys', 'INPUT_2': Output_Ordner+'\\'+'key_value_csv.csv', 'FIELD_2': 'keys',
    'FIELDS_TO_COPY': ['values'], 'METHOD': 1, 'DISCARD_NONMATCHING': False, 'PREFIX': '',
    'OUTPUT': Output_Ordner+'\\'+'result'})