7

I am using different shapefiles from open data in Switzerland. They have just updated the whole set. My question is: how to compared two layers (new and old version)? I would like to know the differences in geometry, as well as attributes. In the data there are points, lines and polygons and some of the sets contains a big amount of features (more than 300 000 elements).

I have already checked all the simillar questions, but they all focus on creating the third layer with differences or particular layer example.

What I want to know is just what has been changed.

I am working with QGIS 2.14.3 and i am familiar with Python.

Improve this question
5
  • Are you looking for a specific way to highlight the difference? as it stands, how would you know what has changed if you don't get a third layer with differences?
    – Hasan Mustafa
    Sep 28, 2016 at 13:08
  • @Hasan Mustafa I do not have in mind any specific way to show it. I just wanted to emphasize that it does not need to be third layer, maybe there is another way to see it. what is more, the difference with third layer shows only changes in geometry, not in attributes.
    – sweet.sugar.cola
    Sep 28, 2016 at 13:52
  • I think you should look into importing the shapefiles into a database like PostgreSQL, with over 300,000 elements most other options will have performance issues.
    – Hasan Mustafa
    Sep 29, 2016 at 5:23
  • @sweet.sugar.cola you are aware of this answer? gis.stackexchange.com/questions/37381/…
    – LarsVegas
    Oct 5, 2016 at 13:12
  • @LarsVegas yes I saw this answer aleady. I know that I can use it for geometric changes, but what was also important for me was checking the attributes' changes. What is more, I have a lot of shapefiles, also big ones, so this approach can be quite long to execute.
    – sweet.sugar.cola
    Oct 12, 2016 at 9:23

3 Answers 3

Reset to default
7

It is not with QGIS or PyQGIS, but if you know Python and the modules Pandas and GeoPandas (Python 2.7 and 3.x), it is easy using the solution of Outputting difference in two pandas dataframes side by side - highlighting the difference if the two shapefiles have the same schema and the same record indexes

import geopandas as gp
# convert shapefiles to GeoDataFrame
old = gp.GeoDataFrame.from_file("shape_old.shp")
new = gp.GeoDataFrame.from_file("shape_new.shp")
old

enter image description here

new

enter image description here

import numpy as np
import pandas as pd
# which entries have changed
ne_stacked = (old != new).stack()
changed = ne_stacked[ne_stacked]
changed.index.names = ['id', 'col']
print changed
id  col     
0   geometry    True
    test        True
1   ensayo      True
2   ensayo      True
    geometry    True

Compare the columns which has been changed.

difference_locations = np.where(old != new)
changed_from = old.values[difference_locations]
changed_to = new.values[difference_locations]
pd.DataFrame({'from': changed_from, 'to': changed_to}, index=changed.index)

enter image description here

But with more than 300 000 elements...

Improve this answer
5
  • 1
    i have never used pandas or geopandas, can you edit and update shapefiles with it?
    – ziggy
    Sep 28, 2016 at 19:51
  • Yes, without problem, GeoPandas is one of the geospatial modules to work in Python only, without GIS.
    – gene
    Sep 28, 2016 at 20:09
  • I have never used Pandas or GeoPandas, but it actually looks interesting. I will take a deeper look inside. So as I understand, just to be clear, I can just load some GIS data into python environment and compare the data there?
    – sweet.sugar.cola
    Sep 29, 2016 at 12:06
  • QGIS data are shapefiles, look at Simple thematic mapping of shapefile using Python? or Python Script examples for geoprocessing shapefiles without using arcpy° for example
    – gene
    Sep 29, 2016 at 15:24
  • Thanks for the code, it works fine when there is no NULL values in your shapefile attributes. But when I try it with NULL attribute values it shows all NULL columns as changed. This NULL data is NULL in both of the shapefiles. Do you have a solution for this? Note: I've tried with dataframe.fillna() but I wasn't successful.
    – Ayda Aktas
    Oct 18, 2016 at 16:17
3

There is a new tool which was added in QGIS 3.12 called "Detect dataset changes" that does exactly what you want. It accepts any geometry type including line. The tool

Compares two vector layers, and determines which features are unchanged, added or deleted between the two. It is designed for comparing two different versions of the same dataset.

Original line:

enter image description here

Modified line:

enter image description here

Changes happened:

enter image description here

Please refer to the help above for more detailed information.

Information in the changelog for QGIS 3.12: https://qgis.org/en/site/forusers/visualchangelog312/#feature-add-new-algorithm-detect-dataset-changes

Improve this answer
0

Maybe a solution is to join the attributes of the same feature (after and before the changes) into one table (you can name added attributes differently) so you can export it to excel and do your math to find differences, %, and so on.

You may even don't need to join those attributes. Just export the attributes of all the features and work the differences in Excel.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.