Convert TIFF file with multiple bands into a single CSV file

Question

I have a GeoTIFF file which contains 13 bands and I want to create one .csv file using gdal_translate, but by default GDAL generates one CSV with one band's data

I want this kind of CSV:

 lat     longi  band1 band2 band3 band4
 2.565   45.545  45    65    34    34   

I want to know if it is possible using GDAL.

gdal_translate -of XYZ -b 4 -b 8 subset_S2A_OPER_MTD_SAFL1C_PDMC_20161017T172323_R019_V20161016T050802_20161016T051302_resampled_b4_b8_reprojected.tif subset_S2A_OPER_MTD_SAFL1C_PDMC_20161017T172323_R019_V20161016T050802_20161016T051302_resampled_b4_b8_reprojected.csv

Here only one band is selected ......

Answer 1

AFAIK gdal_translate is able to convert only one band into XYZ format, so I suggest the following workflow:

1. use gdal_translate to convert each band into a CSV file;
2. use ogr2ogr to append each CSV file to an SQLite file;
3. use ogrinfo to create the index on the ogc_fid field of each SQLite table in order to make the last step fast;
4. finally, join all the tables and get the desired result.

This is a sample batch file where the above steps are implemented for only three bands (change the FILENAME variable and adapt the join query according to your input data):

set FILENAME=test
if exist %FILENAME%.sqlite del %FILENAME%.sqlite
for /L %%i in (1,1,3) do (
gdal_translate -b %%i -of XYZ %FILENAME%.jpg b%%i.csv -co ADD_HEADER_LINE=YES
ogr2ogr -update -append -f SQLite %FILENAME%.sqlite -nln b%%i b%%i.csv -dsco METADATA=NO -dsco INIT_WITH_EPSG=NO
ogrinfo %FILENAME%.sqlite -sql "CREATE INDEX b%%i_ogc_fid_index ON b%%i(ogc_fid)"
)
ogr2ogr -f CSV -sql "SELECT b1.x AS lon, b1.y AS lat, b1.z AS band1, b2.z AS band2, b3.z AS band3 FROM b1 LEFT JOIN b2 ON b1.ogc_fid = b2.ogc_fid LEFT JOIN b3 ON b1.ogc_fid = b3.ogc_fid" %FILENAME%_XYZ.csv %FILENAME%.sqlite -lco SEPARATOR=SPACE

Answer 2

I've had the same issue in my work, and ultimately turned to NetCDF utilities to solve it. I know nothing about NetCDF really, just that I can gdal_translate a raster to that format, and that some of the NetCDF CLI tools can be used to accomplish these "joins".

In my limited testing, the NetCDF approach takes about 40 seconds (most of which is writing the CSV out), while the gdal_translate + sqlite approach takes about 5 and a half minutes.

Here's an example:

import subprocess

import xarray as xr


# convert input tifs to netcdf
subprocess.check_call(['gdal_translate', '-of', 'NetCDF', 'biomass.tif', 'biomass.nc'])
subprocess.check_call(['gdal_translate', '-of', 'NetCDF', 'extent.tif', 'extent.nc'])

# rename bands to match data
subprocess.check_call(['ncrename', '-v', 'Band1,biomass', 'biomass.nc'])
subprocess.check_call(['ncrename', '-v', 'Band1,extent', 'extent.nc'])

# run a netcdf append command to add biomass data to the extent netcdf
subprocess.check_call(['ncks', '-A', 'biomass.nc', 'extent.nc'])

# open extent and then write it directly to CSV
# unfortunately there doesn't seem to be a fast CLI tool for this
ds = xr.open_dataset('extent.nc')
df = ds.to_dataframe().reset_index()
print df.head()

df.to_csv('netcdf_output.csv', index=False)

AFAIK there's no way to write one of these "joined" NetCDFs directly to CSV from the command line, but xarray and pandas can read them and write them fine. Hope this helps!

Answer 3

Use the gdal2xyz.py file

gdal2xyz.py -band 1  -band 2 ... -band n rasterFile.extension   csvFile.csv

using the above command you can convert any rasterFile into a csvFile