Multiple copies of entries after syncing QField to desktop

Question

I am learning my way into qfield and the qfieldsync plugin and I must be doing something wrong.

I have noticed in the desktop project I am getting multiple copies of features which I added in QField when I follow this particular sequence of steps:

I package a project using the plugin. The layer (point-layer)to be edited is set to 'offline editing'. I copy the project files from a qfield-export folder to my phone, open QField, add new features, close qfield, then copy the project files to a different desktop folder 'qfield-import'. In qgis I use Synchronize from QField and the new features are visible. I delete the files from the desktop folder qfield-import. Notice, I do not remove the project files from my phone. Next, I use qfield again, add new features to the point-layer, close QField and then copy the project files from my phone to the desktop qfield-import folder. Run Synchronize from qfield again, the newly added features are visible, but the previously added features are also copied. The pointlayer now has two exact copies of the features added the first time in QField and one copy of the features added in QField the second time.

This might be totally normal behaviour, but it caught me off guard. It seems the qfieldsync plugin does not recognize the first set of points have al ready been synchronized?

The reason I put this question here is I did not find in the documentation anywhere the need to remove the project fies from my phone, do a new packaging after syncing, copy the new export to my phone and then add the second set of points.

Again, it might very well be this is obvious to most, but I was expecting the sync plugin to recognize earlier synced data.

Answer 1

I can confirm problems when importing the same mobile database multiple times to it's main database, if you make changes

What technically happens: When editing data in the mobile project, the changes are logged in the mobile database. On importing it, these changes are appended to the main database according to the log. So if you import the first status, these changes are imported, and if you import the second status, these changes (including the first changes) are imported. So you will get duplicates. Why it behaves differently on PostgreSQL I haven't checked, but I suppose because PostgreSQL is smart enough not to create duplicates.

Anyway. The case to import a mobile database multiple times (without making changes in between) is avoided since QFieldSync checks for the MD5 hash of the mobile database. But this MD5 hash diverts when changes are made on the mobile database meanwhile.

So yes, this should be part of the documentation: Make sure that you synchronize your data back only once. That means, if you go out again to collect more data, you should create a new QField package before to avoid sync problems later on (like e.g. duplicates).

Edit: Btw. I think the upcoming QFieldCloud will provide a nice solution on these workflows.

Answer 2

I tried what you have explained using 2 different options: with GeoPackage as the master Database and with PostgreSQL as the master Database. The results are (if I'm not wrong, since it takes some steps each time):

1. If the master Database is GeoPackage, then the items will be duplicated each time on Synchronize from QField back to master again - that is I can confirm what you observed.
2. If the master is PostgreSQL, then the items will not be duplicated, but updates of attribute values within existing items won't be respected on Synchronize from QField back to master.

The results of both options isn't what one would expect. Maybe your observation was mentioned in issue #45 at the QField repo. If you get no answer here I would suppose you to place an issue to the repo with a link to your issue here. The answer would also interest me.

I have used QField Sync Plugin v4.0.0-beta1 and QGIS 3.18.1 and I have composed the working copies according to the documentation.