Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [precision]

The number of significant digits used to store numbers, particularly coordinate values.

0
votes
0answers
55 views

Losing numeric table precision during shapefile export in ArcGIS Pro

I'm trying to export a GDB file version 10.5 in ArcGIS Pro 2.0.0 to a shapefile, but every time I do, I lose precision with the unique feature IDs. For example the IDs will go from 554423216 --> ...
0
votes
0answers
37 views

Decimal places after converting from point to raster ArcMap

When using "Point to Raster"-Tool in ArcMap I noticed that my point with the elevation of e.g. 128.436m will be converted to a cell with a value of 128.435997. Is it only a display problem with the ...
3
votes
1answer
415 views

How precise is GPS?

Most search results I get on this talk about the accuracy, that is how close the locations outputted by the GPS are to the real positions, of GPS. However, I'm working on a project that only requires ...
2
votes
0answers
29 views

ArcMap Date values lose precision after copying feature class from fGDB to SDE GDB

I have some data in a file geodatabase feature class including some date fields. The precision on the date fields is to milliseconds and this is important. When I copythe feature class from the fGDB ...
1
vote
1answer
82 views

What is the practical limit for lat/long measurement accuracy?

There are several systems for finding lat/long coordinates. This includes GPS satellites, DoD TRANSIT, Russian GLONASS, Galileo, differential GPS, LORAN, land-based systems, and more. Maybe even using ...
1
vote
1answer
34 views

Kriging with variable spatial precision?

I have a data-set comprised of hydraulic conductivity estimates measured at groundwater monitoring wells, as seen here: Exact GPS coordinates are available for some wells, while for others I only ...
1
vote
0answers
120 views

Geohash length to be reversible to LatLong using PostGIS

Decode/encode Geohashes are easy, and there are a "rule of the thumb" for estimat mininal Geohashe length for each LatLong precision. The question is other: precision for revesibility, How many ...
1
vote
0answers
87 views

Nearest point does not intersect line in 'sf' R package

This is a similar question to my recent questions on how to find the nearest point along a polyline and how to split a polyline by an intersecting point using only the sf package. My question has ...
1
vote
0answers
99 views

Changing Virtual field properties in QGis 3.0

I have some existing virtual field with set Length and Precision An example: The formula was first set to rounding it by 2 digits With the current settings it displays 10.00 instead of 10 because of ...
0
votes
2answers
321 views

Accurate GPS apps for smartphones available? [closed]

I'm working on a project where we need to make accurate GPS measurements and the required precision would be in centimeter level. I've been doing some searching and not yet found any solid solutions ...
1
vote
1answer
46 views

Why are short integer fields on SDE auto-updating to Precision 5?

I am having a perceived small issue in Server for ArcGIS (10.3.1 and 10.4.1 on SQL Server) where whenever a Short Integer field is added to a feature class, the Precision auto-updates to Precision: 5 ...
0
votes
0answers
694 views

Shapely and lines intersection

Is there a way in Shapely to calculate intersections of geometries with some tolerance? Imagine we have lines that are the same, but their coordinates are written with different precision. ...
2
votes
1answer
41 views

Precision loss in the REAL fields when saving GRASS layer as shp with QGIS

I am using QGIS version 2.18.14 with GRASS 7.2.2 to clean up some geometry issues within shp files I have. I have an initial shapefile “test.shp” which has a field named “area”. The “area” type is ...
1
vote
0answers
157 views

Postgis st_transform precision

I am checking effects of reprojecting planar coordinates from one CRS to another and back, keeping the same DATUM, and I was trying to determine the cause of the loss of precision. I know that ...
1
vote
1answer
2k views

Join Attributes by Location - Precision

What does "Precision" do within "Join Attributes by Location"? Also, what's the difference between the "Geometric Predicates"?
3
votes
1answer
3k views

Calculate bearing between two decimal GPS coordinates (Arduino / C++)

I am trying to calculate the bearing just like in this example: Calculate bearing between two decimal GPS coordinates , but my environment is Arduino. I got the code translated (please see below), ...
5
votes
1answer
241 views

Does the precision of digitized points vary depending on scale in QGIS?

I have been editing the locations of some points in QGIS. The scale that I am using varies between 1:6000 and 1:12000 depending on the area that I am looking at. I have been seeing some irregularities ...
1
vote
0answers
259 views

define a precision accuracy QGIS project

How to define a precision accuracy of decimal places in QGIS project or where I use digitizing to add new features ? for example I have a point or polygon and true coordinates of vertex have to many ...
2
votes
1answer
60 views

Is there a way to get more than 5 digits of precision when using a double in a QGIS memory layer?

addressLayer = QgsVectorLayer("Point?crs=epsg:4326&field=address:string(50)&field=city:string(50)&field=state:string(50)&field=zipcode:string(50)&field=elevation:double(20,8)&...
8
votes
1answer
2k views

Calculating Optimal Geohash Precision from Bounding Box

I'm using Elasticsearch's GeoHash grid Aggregation to plot clusters on a map (using Leaflet). I understand that for larger areas a lower precision setting should be used to limit the number of buckets ...
1
vote
2answers
162 views

Keeping decimals after spatial location joining using QGIS?

I am merging two layers, a data file and shapefile based on spatial location. I am doing a many to one type join and I want my attribute to take a summary of the intersecting features, using mean ...
1
vote
1answer
67 views

Overriding precision control within Reclassify Tool of ArcGIS Desktop?

I have a coding scheme in which I am trying to apply using the Reclassify tool prior to combining the different raster datasets using raster calculator. I have used the Reclassify tool for other ...
1
vote
0answers
30 views

Is there a measurement standard e.g. for coast lines? [closed]

When watching discovery & documentary shows, coastlines are apparently defined to be just long enough to fit some explanation or a discussed topic. The coastline paradox is a well-established ...
1
vote
0answers
377 views

How to validate high precision GPS systems location [closed]

I am currently working with a high precision RTK-GPS system which claims to be able to reach centimetre level accuracy, I am looking for a way to validate that it is actually reaching that precision ...
4
votes
2answers
438 views

Postgis exact geometry from text

I have some geometry in cgal that I would like to export to postgis. The kernel I'm using is the same as what sfcgal uses (exact predicate exact construction), but I'm encountering some errors in ...
0
votes
1answer
81 views

ArcGIS 10.2.2 Chart Graph data precision shown

I created a pie chart that had several pieces of data; pipe diameter to length of pipe per diameter. I summarized data from length of pipe and diameter from a database and input into a summary data ...
6
votes
1answer
650 views

Force coordinate precision in QGIS when saving feature to PostGIS

Is it possible to configure QGIS in a way, that one can specifically set the desired coordinate precision which is saved for each vertex/point when writing to a PostGIS layer? I want to achieve that ...
1
vote
0answers
1k views

How much is the error in physical longitude and lagitude (e.g. meters) corresponding to a +-0.0001 error in GPS coordinates?

If I have a device how give-me a latitude and longitude with 7 decimals (eg. -42.5124478, -22.3006726, longitude and latitude respectively ) and I round it to 4 decimals (eg. -42.5124, -22.3007). It ...
1
vote
1answer
686 views

Will GPS L2 make RTK more reliable?

I currently experiment with an affordable L1 GPS RTK system based on u-blox NEO-M8T with RTKLIB and higher quality patch antenna (with ground plane). I wonder what is the role of L2 signal in an RTK ...
1
vote
0answers
83 views

Raster Precision QGIS

I am using r.patch extension in QGIS to join a number of LiDAR tiles, some of which overlap hence i am carrying out this task before utilising in a hydraulic model. R.Patch works fine, however the ...
2
votes
0answers
39 views

Do somebody know how factors as “age of data”, “scale” and others influence the precision of GIS data?

Several factors may cause imprecision in datasets. As example of such sources, we should list: "age of data", "formatting", "scale", the technique employed to create the data and others. Each of them ...
1
vote
0answers
88 views

Degree Precision vs. Length [duplicate]

https://en.wikipedia.org/wiki/Decimal_degrees I found this chart in one of the questions here. Additional to this: When making a measurement and getting 2 Measures of Latitude, for example 51....
1
vote
0answers
95 views

FME: floating point precision issues (ArcSDE / FGDB)

I am working on a (generic) FME Workbench that clips and exports data from ArcSDE database into the File Geodatabase (FGDB) format. The SDE features are clipped using an AOI, that arrives in the ...
1
vote
1answer
283 views

Understanding precision of ArcGIS geodatabases?

I would like to know how precision is handled in ArcGIS software. Is it using a real Computer Algebra System engine, an arbitrary precision library or do others tricks to handle precision ?
1
vote
0answers
199 views

How to Change Precision of Export / Add Geometry Columns field

I would like to display the coordinates of a number of points on a map. I can use Vector > Export / Add Geometry Columns which adds coordinates to 6 dp. However as I am using a UTM-type projection ...
0
votes
0answers
473 views

Why is ArcGis change precision of double field after copy?

I create the attribute table of a shapefile and while I copy it by using the "copy features" tool it change the precision from 11 to 14. I create two fields with the name poly_x and poly_y and they ...
1
vote
0answers
654 views

Precision of conversion from WSG84 to ECEF and ECEF to ENU?

To convert from a global to a local coordinate system, I apply the following transformations. WSG84 to ECEG: with and ECEF to ENU: Let's assume that the semi-major axes a and e are known with high ...
1
vote
1answer
60 views

Transfer from Android phone to Windows OS truncates lat long

I am clicking GPS enabled pictures of sites which are less than 500 metres apart through Android based Samsung ACE Duos phone. I later use Google Earth on my Laptop to view these geo tagged pictures. ...
2
votes
2answers
973 views

How to handle coordinates accuracy in ArcGIS

I am creating very basic rectangular polygon using very basic a Python code nCols,nRows=100,100 corners =[[0,0],[0,nRows],[nCols,nRows],[nCols,0],[0,0]] p=arcpy.Polygon(arcpy.Array([arcpy.Point(*...
2
votes
3answers
202 views

Is this distribution of GPS positions caused by multipath distortion?

I have some GPS points that are close to where they should be, but not nearly close enough (horizontal precision is around 10 meters). Looking at the positions which comprise these points (there are ...
0
votes
1answer
690 views

Calculate coordinates with better precision in ArcGIS for Desktop?

I'm using the geometry calculator to get the geographic coordinates of a point shapefile. However when I run it, it calculates the coordinates with 4 decimal points, regardless I've chosen double an ...
3
votes
2answers
114 views

Compensating lat/long offsets after conversions

In my application I am doing conversions between two different formats of data. First format is a simple one containing NE and SW coordinates for a given rectangular area: area = { "sw": { ...
1
vote
1answer
1k views

PyQGIS export to JSON: writeAsVectorFormat and precision

In my QGIS plugin qgis2leaf I am exporting all layers as JSON into a JSON directory. Therefore I would like to enable the user to define the precision in the GUI. The normal "save as" dialogue in QGIS ...
3
votes
1answer
338 views

Row.Value returns rounded SHAPE.area and SHAPE.len values?

I have created a VB.Net Add-In, and am using IRow to get the value of fields in tables in ArcMap to use in a query. But I am encountering a problem when getting the values of the SHAPE.area and ...
17
votes
1answer
22k views

What is the precision of a Geohash

I would like to know the precision of a geohash with a given length. If there is a 'simple' formula you can use to calculate it, that would be extra-cool. Wikipedia lists the precision up to 8 ...
3
votes
4answers
1k views

Densifying JTS LINESTRING (add points)?

I have a LineString I would like to "soften" in a way that if two coordinates (vertices) in the LineString are very far apart, I want to automatically add some coordinates in between. The line ...
11
votes
2answers
19k views

Low-cost centimeter-level accurate satellite positioning (GNSS/GPS) [closed]

I am looking for a low-cost (< US$500) GNSS/GPS setup with a centimeter precision without much HW hacking. I am not able to produce my PCB or do any soldering (though I would do that if there is no ...
1
vote
0answers
258 views

Reduced real-type numbers in QGIS? [closed]

I have a SHP point file created from CSV with QGIS 2.4.0 Chugiak. It contains some data on noise in a certain point. When I import from CSV, it seems to be fine, but when I export to SHP and open the ...
1
vote
0answers
55 views

calculation precision during projection transformation

I am using wgs 84 base tiles, and the tile is not rendered as image but with geojson format. I try to draw them in the tile according to the data. And the geometry of the returned feature is lng and ...
1
vote
1answer
243 views

Unable to select FLOAT precision when adding column

I'm trying to do the lessons in the training-manual at qgis.org. I,m having some problems with 3.3 Ratio Classification http://www.qgis.org/en/docs/training_manual/vector_classification/...