1

I want to access landsat data from gcp-public-data-landsat bucket using vsigs . My code looks like ,

import os, sys
from osgeo import gdal

print(gdal.VersionInfo('–version'))
print(sys.version)

url = "/vsigs/gcp-public-data-landsat/LC08/01/044/034/LC08_L1GT_044034_20130330_20170310_01_T2/LC08_L1GT_044034_20130330_20170310_01_T2_B1.TIF"

gdal.UseExceptions()
gdal.SetConfigOption('CPL_CURL_VERBOSE', 'YES')
gdal.SetConfigOption('CPL_DEBUG', 'YES')
gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS', '.TIF')
gdal.SetConfigOption('GS_SECRET_ACCESS_KEY', '123123')
gdal.SetConfigOption('GS_ACCESS_KEY_ID', 'blablabla')
src = gdal.Open(url)
print(gdal.Info(src))

But it shows

GDAL 2.4.2, released 2019/06/28
3.6.8 (default, Oct  7 2019, 12:59:55) 
[GCC 8.3.0]
Traceback (most recent call last):

  File "<ipython-input-13-5aadfee561e4>", line 1, in <module>
    runfile('/home/souvik/Desktop/Final_Year_Project/google auth/untitled0.py', wdir='/home/souvik/Desktop/Final_Year_Project/google auth')

  File "/usr/lib/python3/dist-packages/spyder/utils/site/sitecustomize.py", line 705, in runfile
    execfile(filename, namespace)

  File "/usr/lib/python3/dist-packages/spyder/utils/site/sitecustomize.py", line 102, in execfile
    exec(compile(f.read(), filename, 'exec'), namespace)

  File "/home/souvik/Desktop/Final_Year_Project/google auth/untitled0.py", line 52, in <module>
    src = gdal.Open(url)

  File "/usr/lib/python3/dist-packages/osgeo/gdal.py", line 3242, in Open
    return _gdal.Open(*args)

RuntimeError: `/vsigs/gcp-public-data-landsat/LC08/01/044/034/LC08_L1GT_044034_20130330_20170310_01_T2/LC08_L1GT_044034_20130330_20170310_01_T2_B1.TIF' does not exist in the file system, and is not recognized as a supported dataset name. 

I also tried using rasterio , but it also shows parsed path problem. Also I have seen https://github.com/OSGeo/gdal/blob/master/autotest/gcore/vsigs.py .But could not figure out. Please tell to acsess it simply using GS_SECRET_ACCESS_KEY and GS_ACCESS_KEY_ID .

  • You haven't included the debug text. If you don't see it, can you try running the code outside spyder (i.e. via the command line). – user2856 yesterday
0

"/vsigs" doesn't use the full URL, just pass /vsigs/gcp-public-data-landsat/path/to/file.tif

For example:

import os, sys
from osgeo import gdal

print(gdal.VersionInfo('–version'))
print(sys.version)

url = "/vsigs/gcp-public-data-landsat/LC08/01/044/034/LC08_L1GT_044034_20130330_20170310_01_T2/LC08_L1GT_044034_20130330_20170310_01_T2_B1.TIF"

gdal.UseExceptions()
gdal.SetConfigOption('CPL_CURL_VERBOSE', 'YES')
gdal.SetConfigOption('CPL_DEBUG', 'YES')
gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS', '.TIF')
gdal.SetConfigOption('GS_SECRET_ACCESS_KEY', '23412341234123412341234')
gdal.SetConfigOption('GS_ACCESS_KEY_ID', 'blahblah')
src = gdal.Open(url)
print(gdal.Info(src))

Output:

GDAL 2.4.0, released 2018/12/14
3.7.3 | packaged by conda-forge | (default, Mar 27 2019, 23:01:00) 

#### DEBUG text ####
*   Trying 2404:6800:4006:809::2010...
* TCP_NODELAY set
* Connected to storage.googleapis.com (2404:6800:4006:809::2010) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: osgeo/ssl/cacert.pem
  CApath: none
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=*.storage.googleapis.com
*  start date: Jan 29 21:59:39 2020 GMT
*  expire date: Apr 22 21:59:39 2020 GMT
*  subjectAltName: host "storage.googleapis.com" matched cert's "*.googleapis.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
> GET /gcp-public-data-landsat/?delimiter=%2F&prefix=LC08%2F01%2F044%2F034%2FLC08_L1GT_044034_20130330_20170310_01_T2%2F HTTP/1.1
Host: storage.googleapis.com
Accept: */*

#### I've truncated the DEBUG text ####

Driver: GTiff/GeoTIFF
Files: /vsigs/gcp-public-data-landsat/LC08/01/044/034/LC08_L1GT_044034_20130330_20170310_01_T2/LC08_L1GT_044034_20130330_20170310_01_T2_B1.TIF
       /vsigs/gcp-public-data-landsat/LC08/01/044/034/LC08_L1GT_044034_20130330_20170310_01_T2/LC08_L1GT_044034_20130330_20170310_01_T2_MTL.txt
Size is 7511, 7421
Coordinate System is:
PROJCS["WGS 84 / UTM zone 10N",
    GEOGCS["WGS 84",
        DATUM["WGS_1984",
            SPHEROID["WGS 84",6378137,298.257223563,
                AUTHORITY["EPSG","7030"]],
            AUTHORITY["EPSG","6326"]],
        PRIMEM["Greenwich",0,
            AUTHORITY["EPSG","8901"]],
        UNIT["degree",0.0174532925199433,
            AUTHORITY["EPSG","9122"]],
        AUTHORITY["EPSG","4326"]],
    PROJECTION["Transverse_Mercator"],
    PARAMETER["latitude_of_origin",0],
    PARAMETER["central_meridian",-123],
    PARAMETER["scale_factor",0.9996],
    PARAMETER["false_easting",500000],
    PARAMETER["false_northing",0],
    UNIT["metre",1,
        AUTHORITY["EPSG","9001"]],
    AXIS["Easting",EAST],
    AXIS["Northing",NORTH],
    AUTHORITY["EPSG","32610"]]
Origin = (332385.000000000000000,4258815.000000000000000)
Pixel Size = (30.000000000000000,-30.000000000000000)
Metadata:
  AREA_OR_POINT=Point
  METADATATYPE=ODL
Image Structure Metadata:
  COMPRESSION=LZW
  INTERLEAVE=BAND
Corner Coordinates:
Upper Left  (  332385.000, 4258815.000) (124d55'16.12"W, 38d27'42.87"N)
Lower Left  (  332385.000, 4036185.000) (124d52'13.76"W, 36d27'22.62"N)
Upper Right (  557715.000, 4258815.000) (122d20'18.02"W, 38d28'32.86"N)
Lower Right (  557715.000, 4036185.000) (122d21'20.87"W, 36d28' 9.14"N)
Center      (  445050.000, 4147500.000) (123d37'17.21"W, 37d28'21.92"N)
Band 1 Block=256x256 Type=UInt16, ColorInterp=Gray
  • It gives error in my case sir. RuntimeError: `/vsigs/gcp-public-data-landsat/LC08/01/044/034/LC08_L1GT_044034_20130330_20170310_01_T2/LC08_L1GT_044034_20130330_20170310_01_T2_B1.TIF' does not exist in the file system, and is not recognized as a supported dataset name. – Souvik Sankar Mitra 2 days ago
  • Please edit your question and run the edited code in my answer (replacing the GS_SECRET_ACCESS_KEY and GS_ACCESS_KEY_ID values with your actual values). Please include the full output of the code, especially including the debug text (please censor your GS_SECRET_ACCESS_KEY and GS_ACCESS_KEY_ID from the debug text). – user2856 2 days ago
  • I have edited my question sir. – Souvik Sankar Mitra 2 days ago

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.