From Wikipedia: In computing, floating point describes a method of representing an approximation to real numbers in a way that can support a wide range of values. The numbers are, in general, represented approximately to a fixed number of significant digits (the mantissa) and scaled using an ...

learn more… | top users | synonyms

2
votes
1answer
46 views

arcpy handling of floating points

This arises from my own question How to handle coordinates accuracy in ArcGIS where I have tried to use the documentation entitled Using geometry objects with geoprocessing tools as a reference. I ...
1
vote
0answers
18 views

gdal_polygonize floating values [duplicate]

gdal_polygonize converts whatever rasters values into vector integers as it is based on the GDALPolygonize() function (http://gdal.org/gdal__alg_8h.html#a3f522a9035d3512b5d414fb4752671b1). It is ...
2
votes
1answer
42 views

arcpy cursor not accepting float(Inf)

I have a vector layer in which I have added some fields of type 'float' using arcpy.AddField_management. I'm performing simple calculations between those fields using update cursors but the problem is ...
0
votes
1answer
122 views

How to round up/down an pixel value of a raster to a pixel value with 2 decimal places?

I have a raster (DTM) with floating points as pixel value. Now i want to round up/down the pixel values (to its next nearest value) to 2 decimal places. I use ArcGIS (with extensions) and tried the ...
1
vote
0answers
62 views

Conversion of Polygon vector files to Raster GRID 8 bit unsigned integer

I am trying to convert a polygon shapefile layer into Raster GRID with 8 bits and which is unsigned integer. I managed to convert the shapefile in ArcGIS with the Arc Toolbox Conversion tools, but the ...
0
votes
0answers
30 views

Simple query not returning values

I have a super simple select by attribute query "Coefficient" = #, the #'s are 0, .2, .3, etc... to 1. The query will select some of the #s but not others. Example) when I query 0,1,.5 the attributes ...
1
vote
1answer
66 views

How to reclass aspect raster to float values

So I have an aspect raster that was created from a DEM. I'm trying to reclassify it using python, but I'm running into some issues. Here's the setup: I need to reclassify values in the aspect raster ...
0
votes
1answer
42 views

How to interpolate integer rasters to floating point?

I have an reclassified raster, I need to convert it to floating point, the "float" tool does not suffice, because I need the output values to be spatially interpolated. For example, at the midpoint of ...
1
vote
0answers
104 views

Why does GeoServer's GetFeature round my numbers, but only for outputFormat=CSV?

When I make a GetFeature request with outputFormat=CSV to my GeoServer v2.4.5 I'm finding that the floating point numbers in my returned data have been rounded. When I make the same request with ...
1
vote
1answer
667 views

Raster Multiplication - Can't Multiply Sequence by Non-Int of Type “Float”

I am trying to multiply a list of GRID raster datasets using a different value for each raster. The value is in the raster name. Some examples are [wse_1979_5,wse_1980_0,wse_1980_5], where 1979_5, ...
1
vote
1answer
366 views

Floating point calculations in QGIS

How do I manage floating point calculations in QGIS? I have carried out 3 consecutive calculations on landsat bands. I end up with 32 bit floating point rasters on which I can't carry out a 4th ...
3
votes
3answers
2k views

How can I reclassify a raster to floating point values?

I have a raster of landuse values that I need to reclassify into pollution concentration values. I have the pollution values in a separate table. They are decimal values (ex. 0.0056), and my landuse ...
6
votes
1answer
586 views

How can I snap lines in PostGIS?

I have a few linestrings that together form a long line. The start- and endpoints of the linestrings are equal with their direct neighbour, but when tested the ST_Distance between any 2 lines is like ...
8
votes
3answers
5k views

How to fully load a raster into a numpy array?

I have been trying to check my filters on DEM raster for pattern recognition and it is always resulting in missing last rows and columns(like..20). I have tried with PIL library, image load. Then with ...
2
votes
4answers
510 views

Floating point: understanding their inaccuracy [closed]

I am trying to understand why a floating point is inaccurate. Please bear in mind that I am not a mathematician. What is exactly the difference between floating point and double integers? and why use ...
19
votes
2answers
437 views

Precision when working with spatial data in GIS

Is there a basic or introductory study that examines and compares the precision of spatial data when working with varying precison of the data input, like having 1, 2, ... decimal places? with the ...
9
votes
1answer
874 views

Referring to NaN or ±∞ (special floating point values) in ArcMap definition queries?

I have by chance discovered how ArcMap displays special floating point values to the user. +∞ (positive infinity) is displayed as 1.#INF –∞ (negative infinity) would supposedly be displayed as ...