Questions tagged [precision]

The number of significant digits used to store numbers, particularly coordinate values.

Filter by
Sorted by
Tagged with
1 vote
0 answers
29 views

Precision of latitude / longitude in history

I am looking to assign a likely precision value for a point based on the decade when the measurement was made. Lets say that in the 1910's the precision widely available would be to two decimal places....
user avatar
1 vote
1 answer
65 views

Why does casting geometry to box2d give different precision from 2 databases

I have a postgresql 9.0 DB with postgis extension that I have migrated to another server running postgresql 11.13. Visual inspection of the tables shows matching data in raw form. E.g.: When ...
user avatar
  • 41
9 votes
2 answers
174 views

Tool for exporting vector layer in QGIS

I am looking for a tool in the toolbox which gives me exactly the options that "Export Layer" is giving me (see screenshot) because I want to use it in a model. For example, I want to cut ...
user avatar
  • 1,903
2 votes
0 answers
36 views

Uncertainty in estimating a polygon area in GEE

I have 2 methods to compute the area of a polygon: Using the dedicated method: ee.Feature.area() Summing the area of each pixels of an image reduced to the polygon using ee.Image.reduceRegion() ...
user avatar
  • 41
1 vote
2 answers
432 views

Change in coordinate precision with ogr2ogr moving from GDAL 2 to 3

I'm migrating some old code from GDAL 2 to GDAL 3. I use ogr2ogr to import a shapefile into PostGIS like this: ogr2ogr -f PostgreSQL -nln mwe PG:"host=localhost user=myuser" mwe.shp -...
user avatar
3 votes
1 answer
235 views

Why can't I change precision of decimal number in new Field Calculator?

Can't click those fields. It worked in the past and now just don't. I can't figure it out. And even it's set to three by default (and I can't change that), it ends up being infinite.
user avatar
5 votes
2 answers
413 views

Trimming decimal numbers for coordinates/heights in QGIS

I have a GeoPackage table with coordinates and heights. I only want 3 decimals for the coordinates and 2 decimals for the heights. In the past I could easily define the precision in the field ...
user avatar
  • 1,903
5 votes
0 answers
29 views

Wrangling precision in QGIS with a hypothetical planet

I'm a tabletop roleplaying gamemaster wanting to use QGIS to help organize my notes according to place. GIS addresses my need to store notes and other data that may be anywhere from a few feet from ...
user avatar
3 votes
0 answers
162 views

Reducing precision of GeoJSON

We have a large number of GeoJSON shapes that are stored in a datastore. Currently, the GeoJSON coordinates are being stored with 15 decimals, which is accurate up to picometers. We don't need that ...
user avatar
  • 161
1 vote
0 answers
145 views

Create attribute field on a geopackage with precision using QGIS 3.4 [closed]

On a QGIS geopckage when I create a new field for decimal numbers, I can only create it as a decimal (real). So it populates the field with many decimals as shown. I don't mind having more decimal ...
user avatar
  • 467
0 votes
1 answer
80 views

Are ECEF coordinates determined by PPP post-processing valid between different receivers and antennae?

I was reading the tutorial for PPP using gpsd, and it emphasized how the position and altitude produced by different receivers can produce wildly different results depending on the datum and ellipsoid ...
user avatar
4 votes
1 answer
2k views

Truncating points precision

I have geopandas with CRS in meters (ITM {'init' :'epsg:2039'}). I converted the geometry from espg:4326 so I have very high precision which I don't require: POINT (177559.4776412862 594546....
user avatar
  • 357
1 vote
2 answers
413 views

Getting more precision with GeoPandas?

Geopandas Reading a shapefile with GeoPandas and printing its total bounds: dataframe = gpd.read_file('example.shp') print(dataframe.total_bounds) Prints: [ 663590.5817 1541419.8307 724630.0589 ...
user avatar
1 vote
1 answer
175 views

Options for precise relative position calculations

I'm investigating an industrial application that has a requirement to calculate with precision the relative distance on the ground between two outdoor points in the following scenario: The first ...
user avatar
  • 119
2 votes
1 answer
184 views

Set default coordinate precision

Is there a way to set the default coordinate precision to 7 for GeoJSON files in QGIS? This would help me save time when I go to save files.
user avatar
  • 629
1 vote
1 answer
66 views

Can exporting geodata to OSM data format with less precision cause self-intersections? How to address it?

I have my geodata layers processed in GRASS GIS. Its internal tools report that the topology has no problems, such as self- and mutual intersections. Then I export the data to a GML file which then ...
user avatar
0 votes
0 answers
113 views

Losing numeric table precision during shapefile export in ArcGIS Pro

I'm trying to export a GDB file version 10.5 in ArcGIS Pro 2.0.0 to a shapefile, but every time I do, I lose precision with the unique feature IDs. For example the IDs will go from 554423216 --> ...
user avatar
  • 629
0 votes
0 answers
272 views

Decimal places after converting from point to raster ArcMap

When using "Point to Raster"-Tool in ArcMap I noticed that my point with the elevation of e.g. 128.436m will be converted to a cell with a value of 128.435997. Is it only a display problem with the ...
user avatar
  • 43
3 votes
1 answer
5k views

How precise is GPS?

Most search results I get on this talk about the accuracy, that is how close the locations outputted by the GPS are to the real positions, of GPS. However, I'm working on a project that only requires ...
user avatar
2 votes
0 answers
96 views

ArcMap Date values lose precision after copying feature class from fGDB to SDE GDB

I have some data in a file geodatabase feature class including some date fields. The precision on the date fields is to milliseconds and this is important. When I copythe feature class from the fGDB ...
user avatar
1 vote
1 answer
452 views

What is the practical limit for lat/long measurement accuracy?

There are several systems for finding lat/long coordinates. This includes GPS satellites, DoD TRANSIT, Russian GLONASS, Galileo, differential GPS, LORAN, land-based systems, and more. Maybe even using ...
user avatar
1 vote
1 answer
53 views

Kriging with variable spatial precision?

I have a data-set comprised of hydraulic conductivity estimates measured at groundwater monitoring wells, as seen here: Exact GPS coordinates are available for some wells, while for others I only ...
user avatar
1 vote
0 answers
425 views

Geohash length to be reversible to LatLong using PostGIS

Decode/encode Geohashes are easy, and there are a "rule of the thumb" for estimat mininal Geohashe length for each LatLong precision. The question is other: precision for revesibility, How many ...
user avatar
  • 2,084
2 votes
0 answers
457 views

Nearest point does not intersect line in 'sf' R package

This is a similar question to my recent questions on how to find the nearest point along a polyline and how to split a polyline by an intersecting point using only the sf package. My question has ...
user avatar
  • 780
2 votes
0 answers
250 views

Changing Virtual field properties in QGis 3.0

I have some existing virtual field with set Length and Precision An example: The formula was first set to rounding it by 2 digits With the current settings it displays 10.00 instead of 10 because of ...
user avatar
  • 191
0 votes
2 answers
1k views

Accurate GPS apps for smartphones available? [closed]

I'm working on a project where we need to make accurate GPS measurements and the required precision would be in centimeter level. I've been doing some searching and not yet found any solid solutions ...
user avatar
  • 347
2 votes
1 answer
272 views

Why are short integer fields on SDE auto-updating to Precision 5?

I am having a perceived small issue in Server for ArcGIS (10.3.1 and 10.4.1 on SQL Server) where whenever a Short Integer field is added to a feature class, the Precision auto-updates to Precision: 5 ...
user avatar
  • 1,242
0 votes
0 answers
2k views

Shapely and lines intersection

Is there a way in Shapely to calculate intersections of geometries with some tolerance? Imagine we have lines that are the same, but their coordinates are written with different precision. ...
user avatar
  • 2,294
2 votes
1 answer
84 views

Precision loss in the REAL fields when saving GRASS layer as shp with QGIS

I am using QGIS version 2.18.14 with GRASS 7.2.2 to clean up some geometry issues within shp files I have. I have an initial shapefile “test.shp” which has a field named “area”. The “area” type is ...
user avatar
  • 31
3 votes
0 answers
458 views

Postgis st_transform precision

I am checking effects of reprojecting planar coordinates from one CRS to another and back, keeping the same DATUM, and I was trying to determine the cause of the loss of precision. I know that ...
user avatar
  • 172
1 vote
1 answer
3k views

Join Attributes by Location - Precision

What does "Precision" do within "Join Attributes by Location"? Also, what's the difference between the "Geometric Predicates"?
user avatar
  • 629
5 votes
1 answer
12k views

Calculate bearing between two decimal GPS coordinates (Arduino / C++)

I am trying to calculate the bearing just like in this example: Calculate bearing between two decimal GPS coordinates , but my environment is Arduino. I got the code translated (please see below), ...
user avatar
4 votes
1 answer
324 views

Does the precision of digitized points vary depending on scale in QGIS?

I have been editing the locations of some points in QGIS. The scale that I am using varies between 1:6000 and 1:12000 depending on the area that I am looking at. I have been seeing some irregularities ...
user avatar
  • 2,224
2 votes
0 answers
964 views

define a precision accuracy QGIS project

How to define a precision accuracy of decimal places in QGIS project or where I use digitizing to add new features ? for example I have a point or polygon and true coordinates of vertex have to many ...
user avatar
  • 1,023
2 votes
1 answer
80 views

Is there a way to get more than 5 digits of precision when using a double in a QGIS memory layer?

addressLayer = QgsVectorLayer("Point?crs=epsg:4326&field=address:string(50)&field=city:string(50)&field=state:string(50)&field=zipcode:string(50)&field=elevation:double(20,8)&...
user avatar
10 votes
1 answer
5k views

Calculating Optimal Geohash Precision from Bounding Box

I'm using Elasticsearch's GeoHash grid Aggregation to plot clusters on a map (using Leaflet). I understand that for larger areas a lower precision setting should be used to limit the number of buckets ...
user avatar
  • 103
1 vote
2 answers
285 views

Keeping decimals after spatial location joining using QGIS?

I am merging two layers, a data file and shapefile based on spatial location. I am doing a many to one type join and I want my attribute to take a summary of the intersecting features, using mean ...
user avatar
1 vote
1 answer
130 views

Overriding precision control within Reclassify Tool of ArcGIS Desktop?

I have a coding scheme in which I am trying to apply using the Reclassify tool prior to combining the different raster datasets using raster calculator. I have used the Reclassify tool for other ...
user avatar
1 vote
0 answers
33 views

Is there a measurement standard e.g. for coast lines? [closed]

When watching discovery & documentary shows, coastlines are apparently defined to be just long enough to fit some explanation or a discussed topic. The coastline paradox is a well-established ...
user avatar
  • 111
1 vote
0 answers
514 views

How to validate high precision GPS systems location [closed]

I am currently working with a high precision RTK-GPS system which claims to be able to reach centimetre level accuracy, I am looking for a way to validate that it is actually reaching that precision ...
user avatar
3 votes
2 answers
818 views

Postgis exact geometry from text

I have some geometry in cgal that I would like to export to postgis. The kernel I'm using is the same as what sfcgal uses (exact predicate exact construction), but I'm encountering some errors in ...
user avatar
1 vote
1 answer
110 views

ArcGIS 10.2.2 Chart Graph data precision shown

I created a pie chart that had several pieces of data; pipe diameter to length of pipe per diameter. I summarized data from length of pipe and diameter from a database and input into a summary data ...
user avatar
  • 13
7 votes
1 answer
1k views

Force coordinate precision in QGIS when saving feature to PostGIS

Is it possible to configure QGIS in a way, that one can specifically set the desired coordinate precision which is saved for each vertex/point when writing to a PostGIS layer? I want to achieve that ...
user avatar
  • 451
1 vote
0 answers
3k views

How much is the error in physical longitude and lagitude (e.g. meters) corresponding to a +-0.0001 error in GPS coordinates?

If I have a device how give-me a latitude and longitude with 7 decimals (eg. -42.5124478, -22.3006726, longitude and latitude respectively ) and I round it to 4 decimals (eg. -42.5124, -22.3007). It ...
user avatar
1 vote
1 answer
915 views

Will GPS L2 make RTK more reliable?

I currently experiment with an affordable L1 GPS RTK system based on u-blox NEO-M8T with RTKLIB and higher quality patch antenna (with ground plane). I wonder what is the role of L2 signal in an RTK ...
user avatar
  • 513
1 vote
0 answers
143 views

Raster Precision QGIS

I am using r.patch extension in QGIS to join a number of LiDAR tiles, some of which overlap hence i am carrying out this task before utilising in a hydraulic model. R.Patch works fine, however the ...
user avatar
  • 105
2 votes
0 answers
48 views

Do somebody know how factors as "age of data", "scale" and others influence the precision of GIS data?

Several factors may cause imprecision in datasets. As example of such sources, we should list: "age of data", "formatting", "scale", the technique employed to create the data and others. Each of them ...
user avatar
2 votes
0 answers
176 views

Degree Precision vs. Length [duplicate]

https://en.wikipedia.org/wiki/Decimal_degrees I found this chart in one of the questions here. Additional to this: When making a measurement and getting 2 Measures of Latitude, for example 51....
user avatar
  • 51
1 vote
0 answers
118 views

FME: floating point precision issues (ArcSDE / FGDB)

I am working on a (generic) FME Workbench that clips and exports data from ArcSDE database into the File Geodatabase (FGDB) format. The SDE features are clipped using an AOI, that arrives in the ...
user avatar
1 vote
1 answer
469 views

Understanding precision of ArcGIS geodatabases?

I would like to know how precision is handled in ArcGIS software. Is it using a real Computer Algebra System engine, an arbitrary precision library or do others tricks to handle precision ?
user avatar
  • 184