Skip to main content

Questions tagged [precision]

The number of significant digits used to store numbers, particularly coordinate values.

Filter by
Sorted by
Tagged with
2 votes
4 answers
128 views

Reducing decimal places of coordinates in vector data using QGIS

I have polygon vector data, some of the polygons are imported from another 3D program (Agisoft Metashape). The imported polygons have up to 12 decimal places in the displayed coordinates (attribute ...
chrissi's user avatar
  • 101
0 votes
0 answers
33 views

Navigating to a point with cm accuracy

I am due to install some monitoring points for a field experiment. I have a GNSS receiver (Arrow 100) and so can record the location of my monitoring points with a high degree of accuracy. I am ...
lauralizzy's user avatar
0 votes
1 answer
57 views

DCELL cell type is set automatically for raster after r.resamp.interp

After interpolation with r.resamp.interp with bilinear method DCELL cell type is set for raster. When I export it with r.out.gdal I get a precision loss error, as I export it as Float32 (PCRaster ...
Nikolay Yasinskiy's user avatar
4 votes
4 answers
223 views

Dropping nearly-identical point locations with GeoPandas

I have a GeoDataFrame (gdf) of several thousand point locations. I want to drop duplicate records from the gdf - that is, records with the same attribute information and the same location. However, ...
neirbom9's user avatar
  • 167
3 votes
2 answers
214 views

ST_ReducePrecision function does not work properly

I am trying to use ST_ReducePrecision to reduce the number of characters, because of the character limit in power bi cells. ST_AsText(ST_ReducePrecision(ST_Transform(geom, 4326), 0.000001)) The result ...
Matheus Clarkson's user avatar
1 vote
0 answers
68 views

Changing number to bigger than 999 in QGIS

In QGIS 3.28 on Windows, there is a shapefile originally created in QGIS that was imported and worked on in ArcGIS. Now, when I reopen the file in QGIS, the real numbers have a width of 18 instead of ...
i.i.k.'s user avatar
  • 1,709
2 votes
0 answers
58 views

Does ArcGIS Pro have a tool like PostGIS ST_ReducePrecision?

Is there a tool in ArcGIS Pro to reduce the precision of a geometry in a dataset, similar to https://postgis.net/docs/manual-dev/ST_ReducePrecision.html? I want to use it on a file geodatabase dataset ...
joosthoek's user avatar
  • 307
3 votes
1 answer
181 views

QGIS input coordinate precision

I imported a point geometry layer into QGIS using a delimited text file (CSV) that contained the latitude and longitude values projected in CRS WGS84-EPSG:4326 with 6 decimal places (e.g. 38.145695). ...
guls's user avatar
  • 33
1 vote
2 answers
123 views

Tell GeoPandas the number of significant figures to use when writing GeoJSON geometries

I have a GeoDataFrame gdf. I write its contents to a GeoJSON: gdf.to_file("example.geojson") which works. However, each geometry is specified to 16 significant figures, which is an overkill ...
zabop's user avatar
  • 2,224
1 vote
1 answer
75 views

How to store a featureclass's Domain, Resolution and Tolerance when migrating to PostGIS?

I'd like to copy a featureclass from an Esri Geodatabase into PostGIS and have it retain the Domain, XY Resolution and XY Tolerance values associated with it. I don't see a table in PostGIS (or in the ...
Kirk Kuykendall's user avatar
1 vote
0 answers
138 views

Precision of latitude / longitude in history

I am looking to assign a likely precision value for a point based on the decade when the measurement was made. Lets say that in the 1910's the precision widely available would be to two decimal places....
D.W. Murray's user avatar
1 vote
1 answer
308 views

Why does casting geometry to box2d give different precision from 2 databases

I have a postgresql 9.0 DB with postgis extension that I have migrated to another server running postgresql 11.13. Visual inspection of the tables shows matching data in raw form. E.g.: When ...
Sean's user avatar
  • 41
9 votes
2 answers
869 views

Tool for exporting vector layer in QGIS

I am looking for a tool in the toolbox which gives me exactly the options that "Export Layer" is giving me (see screenshot) because I want to use it in a model. For example, I want to cut ...
Enzo Baldini's user avatar
  • 2,337
3 votes
0 answers
153 views

Uncertainty in estimating a polygon area in GEE

I have 2 methods to compute the area of a polygon: Using the dedicated method: ee.Feature.area() Summing the area of each pixels of an image reduced to the polygon using ee.Image.reduceRegion() ...
tmth's user avatar
  • 61
1 vote
2 answers
1k views

Change in coordinate precision with ogr2ogr moving from GDAL 2 to 3

I'm migrating some old code from GDAL 2 to GDAL 3. I use ogr2ogr to import a shapefile into PostGIS like this: ogr2ogr -f PostgreSQL -nln mwe PG:"host=localhost user=myuser" mwe.shp -...
Charlie Lefrak's user avatar
5 votes
2 answers
1k views

Why can't I change precision of decimal number in new QGIS Field Calculator?

I can't click those fields. It worked in the past and now it just doesn't. I can't figure it out. And even if it's set to three by default (and I can't change that), it is infinite.
Dyakonov's user avatar
5 votes
2 answers
2k views

Trimming decimal numbers for coordinates/heights in QGIS

I have a GeoPackage table with coordinates and heights. I only want 3 decimals for the coordinates and 2 decimals for the heights. In the past I could easily define the precision in the field ...
Enzo Baldini's user avatar
  • 2,337
5 votes
0 answers
44 views

Wrangling precision in QGIS with a hypothetical planet

I'm a tabletop roleplaying gamemaster wanting to use QGIS to help organize my notes according to place. GIS addresses my need to store notes and other data that may be anywhere from a few feet from ...
MadEmperorYuri's user avatar
4 votes
0 answers
481 views

Reducing precision of GeoJSON

We have a large number of GeoJSON shapes that are stored in a datastore. Currently, the GeoJSON coordinates are being stored with 15 decimals, which is accurate up to picometers. We don't need that ...
Kenneth's user avatar
  • 171
1 vote
0 answers
225 views

Create attribute field on a geopackage with precision using QGIS 3.4 [closed]

On a QGIS geopckage when I create a new field for decimal numbers, I can only create it as a decimal (real). So it populates the field with many decimals as shown. I don't mind having more decimal ...
Raj's user avatar
  • 487
1 vote
1 answer
177 views

Are ECEF coordinates determined by PPP post-processing valid between different receivers and antennae?

I was reading the tutorial for PPP using gpsd, and it emphasized how the position and altitude produced by different receivers can produce wildly different results depending on the datum and ellipsoid ...
Nick Anderegg's user avatar
5 votes
2 answers
5k views

Truncating points precision

I have geopandas with CRS in meters (ITM {'init' :'epsg:2039'}). I converted the geometry from espg:4326 so I have very high precision which I don't require: POINT (177559.4776412862 594546....
Shakedk's user avatar
  • 387
1 vote
3 answers
2k views

Getting more precision with GeoPandas?

Geopandas Reading a shapefile with GeoPandas and printing its total bounds: dataframe = gpd.read_file('example.shp') print(dataframe.total_bounds) Prints: [ 663590.5817 1541419.8307 724630.0589 ...
David1212k's user avatar
1 vote
1 answer
323 views

Options for precise relative position calculations

I'm investigating an industrial application that has a requirement to calculate with precision the relative distance on the ground between two outdoor points in the following scenario: The first ...
deandob's user avatar
  • 119
2 votes
1 answer
471 views

Set default coordinate precision

Is there a way to set the default coordinate precision to 7 for GeoJSON files in QGIS? This would help me save time when I go to save files.
csft306's user avatar
  • 649
1 vote
1 answer
155 views

Can exporting geodata to OSM data format with less precision cause self-intersections? How to address it?

I have my geodata layers processed in GRASS GIS. Its internal tools report that the topology has no problems, such as self- and mutual intersections. Then I export the data to a GML file which then ...
Grigory Rechistov's user avatar
0 votes
0 answers
198 views

Losing numeric table precision during shapefile export in ArcGIS Pro

I'm trying to export a GDB file version 10.5 in ArcGIS Pro 2.0.0 to a shapefile, but every time I do, I lose precision with the unique feature IDs. For example the IDs will go from 554423216 --> ...
csft306's user avatar
  • 649
0 votes
0 answers
361 views

Decimal places after converting from point to raster ArcMap

When using "Point to Raster"-Tool in ArcMap I noticed that my point with the elevation of e.g. 128.436m will be converted to a cell with a value of 128.435997. Is it only a display problem with the ...
PDUY's user avatar
  • 53
3 votes
1 answer
6k views

How precise is GPS?

Most search results I get on this talk about the accuracy, that is how close the locations outputted by the GPS are to the real positions, of GPS. However, I'm working on a project that only requires ...
Nathan Yan's user avatar
2 votes
0 answers
127 views

ArcMap Date values lose precision after copying feature class from fGDB to SDE GDB

I have some data in a file geodatabase feature class including some date fields. The precision on the date fields is to milliseconds and this is important. When I copythe feature class from the fGDB ...
Son of a Beach's user avatar
1 vote
1 answer
678 views

What is the practical limit for lat/long measurement accuracy?

There are several systems for finding lat/long coordinates. This includes GPS satellites, DoD TRANSIT, Russian GLONASS, Galileo, differential GPS, LORAN, land-based systems, and more. Maybe even using ...
William Entriken's user avatar
1 vote
1 answer
59 views

Kriging with variable spatial precision?

I have a data-set comprised of hydraulic conductivity estimates measured at groundwater monitoring wells, as seen here: Exact GPS coordinates are available for some wells, while for others I only ...
Parker Banks's user avatar
1 vote
0 answers
585 views

Geohash length to be reversible to LatLong using PostGIS

Decode/encode Geohashes are easy, and there are a "rule of the thumb" for estimat mininal Geohashe length for each LatLong precision. The question is other: precision for revesibility, How many ...
Peter Krauss's user avatar
  • 2,304
2 votes
0 answers
630 views

Nearest point does not intersect line in 'sf' R package

This is a similar question to my recent questions on how to find the nearest point along a polyline and how to split a polyline by an intersecting point using only the sf package. My question has ...
bcaradima's user avatar
  • 837
2 votes
0 answers
310 views

Changing Virtual field properties in QGis 3.0

I have some existing virtual field with set Length and Precision An example: The formula was first set to rounding it by 2 digits With the current settings it displays 10.00 instead of 10 because of ...
Rubedo's user avatar
  • 329
0 votes
2 answers
1k views

Accurate GPS apps for smartphones available? [closed]

I'm working on a project where we need to make accurate GPS measurements and the required precision would be in centimeter level. I've been doing some searching and not yet found any solid solutions ...
jjepsuomi's user avatar
  • 345
2 votes
1 answer
345 views

Why are short integer fields on SDE auto-updating to Precision 5?

I am having a perceived small issue in Server for ArcGIS (10.3.1 and 10.4.1 on SQL Server) where whenever a Short Integer field is added to a feature class, the Precision auto-updates to Precision: 5 ...
MapHound's user avatar
  • 1,252
0 votes
1 answer
2k views

Shapely and lines intersection

Is there a way in Shapely to calculate intersections of geometries with some tolerance? Imagine we have lines that are the same, but their coordinates are written with different precision. ...
culebrón's user avatar
  • 2,324
2 votes
1 answer
130 views

Precision loss in the REAL fields when saving GRASS layer as shp with QGIS

I am using QGIS version 2.18.14 with GRASS 7.2.2 to clean up some geometry issues within shp files I have. I have an initial shapefile “test.shp” which has a field named “area”. The “area” type is ...
H.M.'s user avatar
  • 31
3 votes
0 answers
677 views

Postgis st_transform precision

I am checking effects of reprojecting planar coordinates from one CRS to another and back, keeping the same DATUM, and I was trying to determine the cause of the loss of precision. I know that ...
Francesco's user avatar
  • 182
1 vote
1 answer
3k views

Join Attributes by Location - Precision

What does "Precision" do within "Join Attributes by Location"? Also, what's the difference between the "Geometric Predicates"?
csft306's user avatar
  • 649
5 votes
1 answer
16k views

Calculate bearing between two decimal GPS coordinates (Arduino / C++)

I am trying to calculate the bearing just like in this example: Calculate bearing between two decimal GPS coordinates , but my environment is Arduino. I got the code translated (please see below), ...
carlosbvz's user avatar
4 votes
1 answer
400 views

Does the precision of digitized points vary depending on scale in QGIS?

I have been editing the locations of some points in QGIS. The scale that I am using varies between 1:6000 and 1:12000 depending on the area that I am looking at. I have been seeing some irregularities ...
Kingfisher's user avatar
  • 2,314
2 votes
0 answers
1k views

define a precision accuracy QGIS project

How to define a precision accuracy of decimal places in QGIS project or where I use digitizing to add new features ? for example I have a point or polygon and true coordinates of vertex have to many ...
jessie jes's user avatar
  • 1,061
2 votes
1 answer
111 views

Is there a way to get more than 5 digits of precision when using a double in a QGIS memory layer?

addressLayer = QgsVectorLayer("Point?crs=epsg:4326&field=address:string(50)&field=city:string(50)&field=state:string(50)&field=zipcode:string(50)&field=elevation:double(20,8)&...
J. DeJesus's user avatar
10 votes
1 answer
7k views

Calculating Optimal Geohash Precision from Bounding Box

I'm using Elasticsearch's GeoHash grid Aggregation to plot clusters on a map (using Leaflet). I understand that for larger areas a lower precision setting should be used to limit the number of buckets ...
Peter's user avatar
  • 203
1 vote
2 answers
337 views

Keeping decimals after spatial location joining using QGIS?

I am merging two layers, a data file and shapefile based on spatial location. I am doing a many to one type join and I want my attribute to take a summary of the intersecting features, using mean ...
Henry Cann's user avatar
1 vote
1 answer
153 views

Overriding precision control within Reclassify Tool of ArcGIS Desktop?

I have a coding scheme in which I am trying to apply using the Reclassify tool prior to combining the different raster datasets using raster calculator. I have used the Reclassify tool for other ...
GeoClimber's user avatar
2 votes
0 answers
43 views

Is there a measurement standard e.g. for coast lines? [closed]

When watching discovery & documentary shows, coastlines are apparently defined to be just long enough to fit some explanation or a discussed topic. The coastline paradox is a well-established ...
Arc's user avatar
  • 121
1 vote
0 answers
576 views

How to validate high precision GPS systems location [closed]

I am currently working with a high precision RTK-GPS system which claims to be able to reach centimetre level accuracy, I am looking for a way to validate that it is actually reaching that precision ...
Laurence Cullen's user avatar