11

At which (xth) decimal place of the coordinate values QGIS defines any set of points is duplicate?

I had been thinking QGIS handles ~15th decimal place; but it was the limitation only because I had been working on Shapefiles mainly.

enter image description here

However, the limit does not apply when I choose another data source, like a temporally scratch layer.

enter image description here

Using a dummy data like below, I made a quick test by two tools, to find the smallest value they can identify differences of coordinates:

  • QGIS geoprocessing: Delete duplicate geometries
  • MMQGIS plugin: Delete Duplicate Geometries

enter image description here

enter image description here

Output by QGIS Delete duplicate geometries is the same as the above input data (all 20 records were preserved), so it may mean QGIS thinks they are all distinct. Does this limit go beyond 1e-29 (or 1e-30) seen in this small test?

Just as comparison, MMQGIS Delete Duplicate Geometries produced below. It seems MMQGIS sets the 16th or 17th decimal places as the limit.

enter image description here

[Edit]

I am afraid I had not been clear what was the central part of my question. My aim to understand the relationship between coordinate values and the duplicates/overlapping, that is as stated in the first paragraph of my question. Hopefully this kind of knowledge can help us easily control overlapping features by tweaking field calculator expressions.

However, underlying issue I had been trying to focus was that the decimal places QGIS recognizes points (nodes) as overlapping seem to vary, depending on the data source.

If we use Shapefiles for our layer, QGIS handles ~15th decimal place and smaller difference (at 16th or 17th) is not recognized... MMQGIS is also affected by this limitation (to my eyes). If we create a temporally scratch layer or DB layers this limitation goes well beyond 30th-ish? This change in behavior makes me wonder.

|improve this question
  • 2
    With ArcGis when creating a feature classes in a database you can specify the minimal difference between two coordinate to be considered different (the default value seem to be 0.001 m for coordinate projected in meter and 0,000000008983153 degree for WGS84). I'm also interested to know what these value are in qgis and if you can modify them. – J.R Apr 17 '18 at 12:30
1

The QGIS Delete Duplicate Geometries tool (in the QGIS 2.18's Geoalgorithms, Vector General Tools) is a Python script that operates on the geometry objects exported/exposed by QGIS' Python layer. And underneath this layer is the GEOS layer.

The GEO layer's geometry equality operator accepts a tolerance parameter (in map distance unit) when comparing two geometries. Unfortunately, this tolerance parameter is not exposed by QGIS' Python layer - hence for Delete Duplicate Geometries, two geometries are considered equal if and only if all their XY values are totally equal. E.g., POINT(1.000000 1.000000) is not equal to POINT(1.00000001 1.00000001).

|improve this answer
  • Thank you for looking into the QGIS geoalgorithm! From your findings, if the wrapper tools (such as Python scripts) do not have access to the tolerance parameter, does it imply that tolerance is solely up to data type? (In other words, is that the reason why it seems as if the tolerance is controlled by the data source we choose?) – Kazuhito Apr 24 '18 at 6:58
  • Yes. At this point in time, you have to truncate or round the X and Y values yourself before doing duplicate detection. This is straightforward for POINTs (although it still involves several steps converting to and fro) but the other geometries will requires (substantial) programming. – Ralph Tee Apr 25 '18 at 5:31
  • I see. Thanks so much for further clarification. It is a bit unfortunate for now, but hopefully future QGIS versions give us (I mean, non-programmers) some more room to control. – Kazuhito Apr 25 '18 at 5:36
1

An interesting reading about this subject is this frequent question: Measuring accuracy of latitude and longitude?

I do not know the tools designed to identify duplicates, but from a computational point of view it is just a matter of the way the numbers are handled by the storage architecture of the entities and the software that handle them. Two different numbers will output FLASE as long as the language represent the numbers in a way that make a bit of difference if the test is if they are not equal. In many cases the interest of duplicate is to find values that are the very same, to find out a kind of typo mistake during digitalization, not how close the are.

In other contexts, the interest of duplicates is to identify objects that are close enough and in practical terms they match.

Assuming the QGIS uses Python as main language, your question can be addressed to the Python docs and you will have an interesting reading about how a base 2 arithmetic differs from our elementary books of maths: Python: Floating Point Arithmetic: Issues and Limitations

|improve this answer
  • Thanks @Marco, but the point I wanted to focus in my question was that the precision (hence directly related to duplicate/overlapping issues) seems to be controlled by its data source (shapefile, or db such as spatialite,...) Sorry it may not have been clear enough. I will edit the question to clarify. – Kazuhito Apr 17 '18 at 10:24
0

I am a registered Land Surveyor in several states. What you see with the different projections are straight lines/distances coming from the Lambert or Mercator Grids. It is like placing a cone on the earth, then the measurement you are taking is placed up on the radii lines above the curvature of the earth. Transverse Mercator is a cylindrical projection for longer East West states like Tennessee. Using survey grade GPS systems we often compare the information with the world projections and even state projections as they lay on the aerials. all measurements are based on the parameters built into the calculations at each point measured. Using GIS Grade equipment that has a sub meter parameter matches very well in one or two mile stretches. After about 6 decimal places in angular measurements is the max you can expect the accuracy to be. No matter how many times you GPS a point with say a one meter accuracy, you will never get a "Mean" or even a "Mode" that will match the high accuracy measurements. To sum it all up, when I am working with Aerials or Satellite information I do not let a meter or so bother me. Oh there are spherical calculations for nap of the earth measurements using theta angles calculating the plumb angle in the field compared to the calculated angle at that point to the center of the earth. There is no such thing as a true measurement. To prove that point to us when I started surveying in 1975 our boss set two nails in hilly road a quarter mile apart and we had to chain it and shoot the distance every day for a month. With atmospheric corrections on the chain and HP distance meter we only matched the original distance 3 times with the distance meter and 1 time with the chain. A very humbling example and stays with me for all these year. i got booed at a GIS convention once when they announced me as a land surveyor (he he) and i had not said a word but I know when to put on my s urveyor hat and when to put on my GIS hat.

|improve this answer
  • I see your point. In fact I have never found myself struglling with real-world accuracy. QGIS is telling me even in the nano-scale or pico-scale I am standing at different location(s), that's also fine. Only I cannot figure out how much decimal place it requires me to put in to say two given points are the duplicates. By the way thanks so much for sharing the story of 1970s, I have never heard of such pioneering phase of mapping and GIS, especially pre-GPS days. Wow. – Kazuhito Apr 23 '18 at 21:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.