Passing GDAL commands in Python using subprocess: errors

Question

Linux Ubuntu 22.04, latest GDAL installation (3.4.1).

Note: skip to 'UPDATE' section further down in this question

gdalinfo don’t recognise Ubuntu file paths or shapefiles:

gdalinfo “/home/username/docs/test.shp”

Leads to this error when run from the default terminal:

ERROR 4: /home/username/docs/test.shp: No such file or directory
gdalinfo failed - unable to open '/home/username/docs/test.shp'.

Strangely a different error appears when Terminal is launched from the directory of the shapefile:

ERROR 4: `/home/username/docs/test.shp' not recognized as a supported file format.
gdalinfo failed - unable to open '/home/username/docs/test.shp'.

Python has no issues recognising the folder paths. For example this returns all files in the folder:

import os
dir = r’/home/username/docs/‘
for file in os.listdir(dir):
    print(file)

returns:

test.shp
test.prj
test.dbf
test.shx

So it appears to be an issue with GDAL. Any ideas?

UPDATE: as pointed out below, I need to use ogrinfo when working with vector data, not gdalinfo which is for raster data. However, I still have an issue referencing an absolute path to a shapefile, only when using gdal in Python (using subprocess call function). For example, using ogr2ogr in Terminal, this works:

ogr2ogr -f "ESRI Shapefile" /home/data/new.shp /home/data/old.shp

but in python this doesn't work:

from subprocess import call
command = r'ogr2ogr -f "ESRI Shapefile" /home/data/new.shp /home/data/old.shp'
call(command)

returns this error:

FileNotFoundError: [Errno 2] No such file or directory: 'oogr2ogr -f "ESRI Shapefile" /home/data/new.shp /home/data/old.shp'

(I'm using dummy paths for this example) From the error it looks like the entire gdal command is being used as a directory??

Answer 1

If its a relative path, you need a period in front of the file path, such as

gdalinfo './Path/to/my/file.shp'

or for the whole path you would specify with tilde such as

gdalinfo '~/home//Path/to/my/file.shp'

so for you if its a relative path it should be

gdalinfo './home/username/docs/test.shp'

Your bigger problem is that gdalinfo is for raster data

So you should actually be using ogrinfo instead, such as:

ogrinfo -so ./home/username/docs/test.shp

Answer 2

It turns out the call method of subprocess doesn't work as well on Linux machines vs Windows. This script has worked for me before (on a Windows setup, with python 3.8):

from subprocess import call

command = r'ogr2ogr -f "ESRI Shapefile" C:/data/new.shp C:/data/old.shp'
call(command)

I've now found that even when swapping the file paths for Linux friendly versions, call would treat the entire command as one long directory when I run this on an Ubuntu machine. My solution (from mightypile's answer here) is to use subprocess.run on a list instead:

import subprocess

subprocess.run(["ogr2ogr", "-f", "ESRI Shapefile", f"/home/data/new.shp", f"/home/data/old.shp"])

Notice how I didn't have to wrap "ESRI Shapefile" in additional single quotes. Also note the absolute paths.

In summary, I'm not sure if something has changed in the subprocess library since I last played with using python to pass gdal commands, or perhaps I had to try different approach on a Linux machine vs my previous Windows machines.