7

I am using different shapefiles from open data in Switzerland. They have just updated the whole set. My question is: how to compared two layers (new and old version)? I would like to know the differences in geometry, as well as attributes. In the data there are points, lines and polygons and some of the sets contains a big amount of features (more than 300 000 elements).

I have already checked all the simillar questions, but they all focus on creating the third layer with differences or particular layer example.

What I want to know is just what has been changed.

I am working with QGIS 2.14.3 and i am familiar with python.

  • Are you looking for a specific way to highlight the difference? as it stands, how would you know what has changed if you don't get a third layer with differences? – Hasan Mustafa Sep 28 '16 at 13:08
  • @Hasan Mustafa I do not have in mind any specific way to show it. I just wanted to emphasize that it does not need to be third layer, maybe there is another way to see it. what is more, the difference with third layer shows only changes in geometry, not in attributes. – sweet.sugar.cola Sep 28 '16 at 13:52
  • I think you should look into importing the shapefiles into a database like PostgreSQL, with over 300,000 elements most other options will have performance issues. – Hasan Mustafa Sep 29 '16 at 5:23
  • @sweet.sugar.cola you are aware of this answer? gis.stackexchange.com/questions/37381/… – LarsVegas Oct 5 '16 at 13:12
  • @LarsVegas yes I saw this answer aleady. I know that I can use it for geometric changes, but what was also important for me was checking the attributes' changes. What is more, I have a lot of shapefiles, also big ones, so this approach can be quite long to execute. – sweet.sugar.cola Oct 12 '16 at 9:23
7

It is not with QGIS or PyQGIS, but if you know Python and the modules Pandas and GeoPandas (Python 2.7 and 3.x), it is easy using the solution of Outputting difference in two pandas dataframes side by side - highlighting the difference if the two shapefiles have the same schema and the same record indexes

import geopandas as gp
# convert shapefiles to GeoDataFrame
old = gp.GeoDataFrame.from_file("shape_old.shp")
new = gp.GeoDataFrame.from_file("shape_new.shp")
old

enter image description here

new  

enter image description here

import numpy as np
import pandas as pd
# which entries have changed
ne_stacked = (old != new).stack()
changed = ne_stacked[ne_stacked]
changed.index.names = ['id', 'col']
print changed
id  col     
0   geometry    True
    test        True
1   ensayo      True
2   ensayo      True
    geometry    True

Compare the columns which has been changed.

difference_locations = np.where(old != new)
changed_from = old.values[difference_locations]
changed_to = new.values[difference_locations]
pd.DataFrame({'from': changed_from, 'to': changed_to}, index=changed.index)

enter image description here

But with more than 300 000 elements...

  • 1
    i have never used pandas or geopandas, can you edit and update shapefiles with it? – ziggy Sep 28 '16 at 19:51
  • Yes, without problem, GeoPandas is one of the geospatial modules to work in Python only, without GIS. – gene Sep 28 '16 at 20:09
  • I have never used Pandas or GeoPandas, but it actually looks interesting. I will take a deeper look inside. So as I understand, just to be clear, I can just load some GIS data into python environment and compare the data there? – sweet.sugar.cola Sep 29 '16 at 12:06
  • Thanks for the code, it works fine when there is no NULL values in your shapefile attributes. But when I try it with NULL attribute values it shows all NULL columns as changed. This NULL data is NULL in both of the shapefiles. Do you have a solution for this? Note: I've tried with dataframe.fillna() but I wasn't successful. – Ayda Aktas Oct 18 '16 at 16:17
0

Maybe a solution is to join the attributes of the same feature (after and before the changes) into one table (you can name added attributes differently) so you can export it to excel and do your math to find differences, %, and so on.

You may even don't need to join those attributes. Just export the attributes of all the features and work the differences in Excel.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.