0

I'm trying to access a web resource through the Python subprocess module and ogr2ogr/ogrinfo. When calling ogr through the command line, everything works fine. When calling with subprocess I have an error. I finally discovered that ogr calls the configuration files from the ArcGIS Pro folder (I'm using conda environment that I cloned from ArcPy) even though I specify using ogrinfo that is installed in the OSGeo4W folder.

Anyway to override this (temporarily, just before the call and then reset it afterwards) ? I guess I have to set GDAL_CONFIG_FILE but I'm not sure how exactly to do this?

On Windows 10.

Command line: C:\OSGeo4W\bin\ogrinfo -ro -al -so --debug ON "https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson" gives:

GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\gdal_ECW_JP2ECW.dll using GDALRegister_ECW_JP2ECW.
GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\gdal_GEOR.dll using GDALRegister_GEOR.
GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\gdal_MG4Lidar.dll using GDALRegister_MG4Lidar.
GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.
GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\ogr_OCI.dll using RegisterOGROCI.
GDAL: Auto register C:\OSGeo4W\bin\gdalplugins\ogr_SOSI.dll using RegisterOGRSOSI.
HTTP: Fetch(https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson)
HTTP: libcurl/7.75.0 OpenSSL/1.1.1o zlib/1.2.12 WinIDN
...

Calling with subprocess

ogrinfoPath = os.path.join("C:"+os.sep, "OSGeo4W", "bin", "ogrinfo.exe") 
args = [ogrinfoPath, "-ro", "-al", "-so", "--debug", "ON", "https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson"]
cmd = subprocess.run(args)

gives:

CPL: Loading configuration from C:\Program Files\ArcGIS\Pro\bin\gdalrc
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_ECW.dll using GDALRegister_ECW.  
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_GRIB.dll using GDALRegister_GRIB.
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_HDF4.dll using GDALRegister_HDF4.
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_HDF5.dll using GDALRegister_HDF5.
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_JP2KAK.dll using GDALRegister_JP2KAK.  
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_JPEGLS.dll using GDALRegister_JPEGLS.  
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_JPIPKAK.dll using GDALRegister_JPIPKAK.
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_KEA.dll using GDALRegister_KEA.        
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_MrSID.dll using GDALRegister_MrSID.    
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_netCDF.dll using GDALRegister_netCDF.  
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_NITF.dll using GDALRegister_NITF.
GDAL: Auto register C:\Program Files\ArcGIS\Pro\bin\gdalplugins\gdal_PDF.dll using GDALRegister_PDF.
HTTP: Fetch(https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson)
HTTP: libcurl/7.75.0 OpenSSL/1.1.1o zlib/1.2.12 WinIDN
Warning 6: HTTP/2 not available in this build of Curl. It needs to be built against nghttp2
Warning 6: GDAL_HTTP_USE_CAPI_STORE requested, but libcurl too old, non-Windows platform or OpenSSL missing.    
HTTP: These HTTP headers were set: Accept: text/plain, application/json
HTTP: Fetch(https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson)
Warning 6: GDAL_HTTP_USE_CAPI_STORE requested, but libcurl too old, non-Windows platform or OpenSSL missing.
HTTP: These HTTP headers were set: Accept: text/plain, application/json
HTTP: Fetch(https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson)
Warning 6: GDAL_HTTP_USE_CAPI_STORE requested, but libcurl too old, non-Windows platform or OpenSSL missing.
HTTP: These HTTP headers were set: Accept: text/plain, application/json
CouchDB: GET /_all_dbs
HTTP: Establish persistent session named 'CouchDB:0000021D12671860'.
HTTP: Fetch(https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson/_all_dbs)
HTTP: These POSTFIELDS were sent:
HTTP: These HTTP headers were set: Content-Type: application/json
HTTP: Ended persistent session named 'CouchDB:0000021D12671860'.
HTTP: Fetch(https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson)
Warning 6: GDAL_HTTP_USE_CAPI_STORE requested, but libcurl too old, non-Windows platform or OpenSSL missing.    
FAILURE:
Unable to open datasource `https://data.montreal.ca/dataset/b628f1da-9dc3-4bb1-9875-1470f891afb1/resource/92cb062a-11be-4222-9ea5-867e7e64c5ff/download/limites-terrestres.geojson' with the following drivers.
  -> ESRIC
  -> PCIDSK
  -> netCDF
  -> PDS4
...

1 Answer 1

2

To set the GDAL_CONFIG_FILE environment variable for only that process you can pass the env arg to subprocess.run().

This expects a dict containing the complete set of vars to be used, so you should pass the entire contents of os.environ with the relevant vars added/modified. Something like this:

import os
import subprocess

env = os.environ.copy()
env['GDAL_CONFIG_FILE'] = r'C:\example\gdalrc'

subprocess.run(['gdalinfo', '--version'], env=env)
1
  • 1
    So there isn't a gdalrc (GDAL_CONFIG_FILE) in my OSGeo4W installation, just passed an empty string: env['GDAL_CONFIG_FILE'] = "" in the os.environ.copy() and it works! I also noticed that the GDAL_DRIVER_PATH points to the ArcGIS folder as well, but didn't need to change it (this time!).
    – mikibok
    Feb 28, 2023 at 14:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.