Questions tagged [floating-point]

From Wikipedia: In computing, floating point describes a method of representing an approximation to real numbers in a way that can support a wide range of values. The numbers are, in general, represented approximately to a fixed number of significant digits (the mantissa) and scaled using an exponent. The base for the scaling is normally 2, 10 or 16.

Filter by
Sorted by
Tagged with
2 votes
1 answer
148 views

Translating raster from floating point to int16 increases size

I know that floating point takes 4 bytes and signed integer 16 takes 2 bytes. I have many DEM .tif rasters, they are all of type floating32. I want to convert them to type integer16 (regardless of ...
Khaled's user avatar
  • 287
0 votes
1 answer
186 views

View a single band floating-point GeoTIFF (generated with GDAL) in QGIS

I am a novice with GIS and geospatial data, so I may be missing something basic. I am working on a process which takes an input PNG, performs a calculation on each input pixel and writes the result to ...
vquixote's user avatar
1 vote
1 answer
777 views

Converting float32 format raster in integer type raster

To load my landscape map on Graphab, my raster needs to be in the format "integer type". However, it is in float32 format and I can't manage to convert it. I am using QGIS. Can someone help ...
Justine J's user avatar
4 votes
1 answer
243 views

Why is this polygon not planar for PostGIS

Why is the following Polygon Z not planar for PostGIS 3.4.0dev (3.3.0rc2-335-gae95dd219)? SELECT ST_IsPlanar( ST_GeomFromText( 'POLYGON Z (( 3021241.9043000005 2037823.3759000003 262....
swiss_knight's user avatar
  • 10.1k
-2 votes
1 answer
170 views

Float data type data shows null in attribute table [duplicate]

When I tried to open table from MS SQL Server, float data type data shows null in the attribute table. It only shows only one record.
Saidatta's user avatar
1 vote
1 answer
164 views

QGIS can not read float from SQL view [closed]

Geometry points from the SQL view view_monitoring with columns from table table_campaign, table_instrument etc. are shown in a QGIS-project. It works fine as long as no column of type float is ...
stonebe's user avatar
  • 445
0 votes
1 answer
327 views

Google Earth Engine scaling behavior of discrete vs continuous image data

I have a simple question related to how GEE determines continuous vs discrete data, specifically during scaling operations as described here. I am unable to find a proper description of this, but my ...
Fernando Moyano's user avatar
0 votes
1 answer
81 views

Trouble Converting gdal_calc.py output to Floating Point

I am new to coding. I'm looking to create an NDVI on S2 data using gdal_calc.py in a Jupyter Notebook. I've downloaded imagery from GEE using code provided by the JRC, European Commission. I selected ...
OpenGeo's user avatar
0 votes
0 answers
60 views

High-resolution Lunar Topography import into QGIS. Float img format?

I don't know how to import these float img files into QGIS https://pgda.gsfc.nasa.gov/products/54 Ultimately I want to be able to export them as 16bit PNG's but I'm stuck now knowing how to import ...
James's user avatar
  • 103
0 votes
1 answer
1k views

Float point problems using Shapely using WKT

I am having float point problems using the Shapely library. I realize now, (after reading the instructions) that using pt.wkt will trim the variable therefore, I am changing my code to uses wkt.dumps(...
Nicolas Cadieux's user avatar
3 votes
3 answers
656 views

Converting floating point raster to vector

I'm trying to convert a raster layer to vector but I'm really struggling. First it says that I have floating points rather than integers so I have I have seen you can use raster calculator to multiply ...
Charlotte Willis's user avatar
0 votes
0 answers
191 views

Raster formats that support colormaps for floating-point data in GDAL?

I am writing a program to overlay a raster image over a map in a KDE Marble project. I am using a combination of QGIS and GDAL to convert non-standard NetCDF data to a raster file. I began using ...
skew_t_pie's user avatar
0 votes
1 answer
693 views

Serving 16 bit floating point with GeoServer?

I gave created a GeoTIFF file with python/gdal as 16 bit floating point (to save space), using -999 as noData value. In GeoServer (v2.15.2), I have defined an SLD color style for this layer. However, ...
Jvg's user avatar
  • 143
1 vote
2 answers
2k views

Exporting a GeoTIFF in Float16 format

I'm trying to merge large raster files (DEM's) together but I run into memory errors using the 'Raster Merge' tool in QGIS 3.8. I suspect the issue is mainly the high amount of precision in the ...
Matt Canada's user avatar
1 vote
0 answers
392 views

Geopackage vector data type

I am reading a shapefile in Python using geopandas, join some tabular data and export to geopackage using geopandas. The entire process is very easy. However, when I export the vector data, the file ...
RutgerH's user avatar
  • 3,265
1 vote
0 answers
268 views

FloatingPointError: invalid value encountered in greater_equal in Semi-Automatic Classification Plugin in QGIS

2019-04-05T00:29:33 WARNING Traceback (most recent call last): File "C:/Users/ASUS/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\SemiAutomaticClassificationPlugin\dock\scpdock.py", line ...
Amanda De Los Santos's user avatar
6 votes
0 answers
211 views

Robust formula to get row,column of raster given X,Y

I'm looking for a robust method to get the row and column of a pixel in a raster, given the coordinates of the origin and the cell spacing. This seems like a problem with an incredibly obvious ...
dbaston's user avatar
  • 13k
0 votes
0 answers
171 views

Any way to use the range slider in ArcGIS Pro with a floating point raster?

So I've created these classified floating point rasters showing specific heights. However, I need to enable range properties for them so that they can function with the range slider in ArcGIS Pro. I ...
Daniel Katleman's user avatar
2 votes
1 answer
481 views

Converting each cell value of raster into floating-point representation using QGIS?

I have several bands of raster from the Landsat8 satellite, and I need to convert them into a floating-point representation. In ArcMap this is done through: Spatial Analyst Toolbox -> Math -> ...
Alexander B's user avatar
1 vote
0 answers
178 views

Feature class - HEC-GeoRAS

I am currently working with HEC-GeoRAS I am trying to import data from a shapefile into the feature class River. I would like to insert a type of data DOUBLE in the attribute ARCLENGTH of the table ...
Sss's user avatar
  • 219
3 votes
1 answer
241 views

Postgis floating point error? in st_split

I am not able to st_split a line with a point on it. I have a linestring and a point in epsg:3857 coordinate system. Their distance is 0: select st_distance( st_geomfromtext('LineString (2121053....
csandor's user avatar
  • 243
1 vote
0 answers
103 views

Raster calculator ArcGIS, high precision floating point operation

I need to preform a transformation on a DTM, in order to use it as a variable in a multiple regression equation (the variables, including elevation data have been box-cox transformed). A part of the ...
MaartenVL's user avatar
1 vote
0 answers
617 views

How to convert raster DEM to reduce file size in QGIS? e.g. 32 to 8 bit floating point

I have a raster DTM (.tif, 32 bit floating point) which has a very large file size and is thus slowing down the software. I am looking for a way of reducing the file size without reducing grid cell ...
Michael_01's user avatar
4 votes
1 answer
422 views

Need less significant digits returned in ArcGIS REST Services

In ArcGIS REST Services Directory when querying a service the values that are set as floats or doubles (elevation, lat, long) return with too many significant digits if the format is JSON. For example ...
jaw485's user avatar
  • 59
2 votes
1 answer
702 views

Convert double to string with Calculator

I would like to convert double to string using the calculator (or ArcObjects) and get the actual number. Calculator seems to truncate the number at ~15 decimal places but the actual value may be a ...
Jakub Sisak GeoGraphics's user avatar
12 votes
6 answers
17k views

Rounding all coordinates in shapely?

Following on from How can I add points to a LineString in shapely?, I'm hitting a problem where floating point representations mean objects aren't precisely where I expect them be after they are moved ...
Jamie Bull's user avatar
3 votes
2 answers
5k views

Creating flt (floating point raster) with QGIS

I would like to create a flt (floating point raster) with QGIS to use later as a terrain in Hec RAS RAS-Mapper. With Raster -> Conversion -> Translate there is no option to create a flt. Is there ...
Isi's user avatar
  • 43
1 vote
1 answer
58 views

Any open source raster applications that accept floating point rasters with attribute tables?

I understand that Arc floating point rasters don't have attribute tables because of the high potential number of unique values in each cell but at a theoretical level, I think a floating point raster ...
Stu Smith's user avatar
  • 8,192
8 votes
3 answers
6k views

Getting ArcMap to recognize csv column as float and not int? [duplicate]

I am trying to show a .csv file (2 columns, 291 rows) in ArcMap 10.2 using Catalog window to drag the .csv file into ArcMap. The second column of the .csv is a floating point number (range is [0-1]), ...
user2489854's user avatar
2 votes
1 answer
187 views

arcpy handling of floating points

This arises from my own question How to handle coordinates accuracy in ArcGIS where I have tried to use the documentation entitled Using geometry objects with geoprocessing tools as a reference. I ...
FelixIP's user avatar
  • 22.8k
1 vote
0 answers
453 views

gdal_polygonize floating values [duplicate]

gdal_polygonize converts whatever rasters values into vector integers as it is based on the GDALPolygonize() function (http://gdal.org/gdal__alg_8h.html#a3f522a9035d3512b5d414fb4752671b1). It is ...
lobelius's user avatar
  • 153
2 votes
1 answer
186 views

arcpy cursor not accepting float(Inf)

I have a vector layer in which I have added some fields of type 'float' using arcpy.AddField_management. I'm performing simple calculations between those fields using update cursors but the problem is ...
Usman's user avatar
  • 309
2 votes
2 answers
9k views

How to round up/down an pixel value of a raster to a pixel value with 2 decimal places?

I have a raster (DTM) with floating points as pixel value. Now i want to round up/down the pixel values (to its next nearest value) to 2 decimal places. I use ArcGIS (with extensions) and tried the ...
Reen's user avatar
  • 189
2 votes
0 answers
428 views

Conversion of Polygon vector files to Raster GRID 8 bit unsigned integer

I am trying to convert a polygon shapefile layer into Raster GRID with 8 bits and which is unsigned integer. I managed to convert the shapefile in ArcGIS with the Arc Toolbox Conversion tools, but the ...
Juwairia's user avatar
1 vote
1 answer
572 views

How to reclass aspect raster to float values

So I have an aspect raster that was created from a DEM. I'm trying to reclassify it using python, but I'm running into some issues. Here's the setup: I need to reclassify values in the aspect raster ...
GISUser9's user avatar
  • 815
0 votes
1 answer
271 views

How to interpolate integer rasters to floating point?

I have an reclassified raster, I need to convert it to floating point, the "float" tool does not suffice, because I need the output values to be spatially interpolated. For example, at the midpoint of ...
jc22's user avatar
  • 425
1 vote
0 answers
680 views

Why does GeoServer's GetFeature round my numbers, but only for outputFormat=CSV?

When I make a GetFeature request with outputFormat=CSV to my GeoServer v2.4.5 I'm finding that the floating point numbers in my returned data have been rounded. When I make the same request with ...
rtanikel's user avatar
3 votes
1 answer
3k views

Raster Multiplication - Can't Multiply Sequence by Non-Int of Type "Float"

I am trying to multiply a list of GRID raster datasets using a different value for each raster. The value is in the raster name. Some examples are [wse_1979_5,wse_1980_0,wse_1980_5], where 1979_5, ...
dubch87's user avatar
  • 337
1 vote
1 answer
3k views

Floating point calculations in QGIS

How do I manage floating point calculations in QGIS? I have carried out 3 consecutive calculations on landsat bands. I end up with 32 bit floating point rasters on which I can't carry out a 4th ...
BillyWill's user avatar
  • 173
5 votes
3 answers
13k views

How can I reclassify a raster to floating point values?

I have a raster of landuse values that I need to reclassify into pollution concentration values. I have the pollution values in a separate table. They are decimal values (ex. 0.0056), and my landuse ...
Tanner's user avatar
  • 7,365
7 votes
2 answers
3k views

How can I snap lines in PostGIS?

I have a few linestrings that together form a long line. The start- and endpoints of the linestrings are equal with their direct neighbour, but when tested the ST_Distance between any 2 lines is like ...
mrg's user avatar
  • 1,565
37 votes
4 answers
75k views

Fully load raster into a numpy array?

I have been trying to check my filters on DEM raster for pattern recognition and it is always resulting in missing last rows and columns(like..20). I have tried with PIL library, image load. Then with ...
najuste's user avatar
  • 817
3 votes
4 answers
1k views

Floating point: understanding their inaccuracy [closed]

I am trying to understand why a floating point is inaccurate. Please bear in mind that I am not a mathematician. What is exactly the difference between floating point and double integers? and why use ...
Robert Buckley's user avatar
22 votes
2 answers
688 views

Precision when working with spatial data in GIS

Is there a basic or introductory study that examines and compares the precision of spatial data when working with varying precison of the data input, like having 1, 2, ... decimal places? with the ...
oschrenk's user avatar
  • 603
10 votes
1 answer
3k views

Referring to NaN or ±∞ (special floating point values) in ArcMap definition queries?

I have by chance discovered how ArcMap displays special floating point values to the user. +∞ (positive infinity) is displayed as 1.#INF –∞ (negative infinity) would supposedly be displayed as -1.#...
stakx's user avatar
  • 1,161