How to Iterate through files in a folder QGIS, using python

Question

I am using QGIS 3.16, and I am trying to run the “service area from layer” function on approximately 400 layers (200,000 starting points in total), I am also due to be repeating this at a range of distances so looking to use python. Also, I cannot merge together as there are 1,000s km of road and just not feasible to set it running for next month.

I have been trying to run the “service are from layer” function via python and I can get it to accept my road and node layer, which have to have the same name (but are in different folders), and then output this to a different folder. The issue is that the table contains a row for each service area, but there is no geometry written. Below is the code that I have been working on but reading about I can see that there is something relating to FeatureSinks but not sure how I go about including this.

I have tried, but I think my Python skills are just a little too rusty.

My code is below.

import os
import processing
import glob
import string

#Folder where my road networks are stored
Select_folderRoads='//Mac/Home/FINAL/March2021/r1/'
#Folder where the starting points/nodes are stored
Select_folderSDH='//Mac/Home/FINAL/March2021/d1/'
#Folder where the output is to be stored
Save_results='//Mac/Home/FINAL/March2021/path250m/'
os.chdir(Select_folderRoads) 

for lyr in glob.glob("*.sqlite"):
        
        #Individual road network layer
        inlayer = Select_folderRoads +lyr
        #Ouptut filename
        outname = Save_results+lyr
        #Layer containing start points/nodes
        startname = Select_folderSDH+lyr
        inlay = lyr[:-7]
        inlayc = inlay.upper()
        
        testinlayer = str(inlayer+'|'+"layername="+inlayc)

        processing.run("native:serviceareafromlayer", 
            {'LAYERS':  inlayer,
            'DEFAULT_DIRECTION': 2,
            'DEFAULT_SPEED': 50.0,
            'DIRECTION_FIELD': '',
            'INCLUDE_BOUNDS': False,
            'INPUT': testinlayer,
            'OUTPUT_LINES':outname,
            'START_POINTS':startname,
            'STRATEGY': 0,
            'TOLERANCE': 0.0,
            'TRAVEL_COST': 250.0,
            'VALUE_BACKWARD': '',
            'VALUE_BOTH': '',
            'VALUE_FORWARD': ''})

Answer 1

I tried to create a working Python code and I considered that you have many SQLite DBs with just one layer (same name as the DB file in lowercase) in your folders r1 and d1.

Your databases are SQLite DB not spatially aware and I loaded the layers with ogr and assign a projection, in your case the OSGB 1936 / British National Grid EPSG: 27700, to tell what coordinate reference system (CRS) layers have.

Once QGIS know the projections, it can do the Service area processing and export geometries.

Here is my working code under QGIS 3.16 (Windows, but I took your question OSX paths for the code below) :

import os
import processing
import glob
import string

#Folder where my road networks are stored
Select_folderRoads='//Mac/Home/FINAL/March2021/r1/'
#Folder where the starting points/nodes are stored
Select_folderSDH='//Mac/Home/FINAL/March2021/d1/'
#Folder where the output is to be stored
Save_results='//Mac/Home/FINAL/March2021/path250m/'
# Set working directory to roads folder
os.chdir(Select_folderRoads)

uk_crs = QgsCoordinateReferenceSystem(27700)

# iterate over roads DB network layers
for dbname in glob.glob("*.sqlite"):
    # get layer name from SQLite DB name
    lyr_name = dbname.split(".sqlite")[0].lower()
    # get roads and points layers, set their CRS
    roads_layer = QgsVectorLayer(f"{Select_folderRoads}{dbname}|layername={lyr_name}", "roads_lyr", "ogr")
    roads_layer.setCrs(uk_crs)
    points_layer = QgsVectorLayer(f"{Select_folderSDH}{dbname}|layername={lyr_name}", "points_lyr", "ogr")
    points_layer.setCrs(uk_crs)

    # processing algorithm that output result as GPKG
    result = processing.run(
        "native:serviceareafromlayer",
        {'DEFAULT_DIRECTION': 2,
        'DEFAULT_SPEED': 50.0,
        'DIRECTION_FIELD': '',
        'INCLUDE_BOUNDS': False,
        'INPUT': roads_layer,
        'OUTPUT_LINES': f"{Save_results}{lyr_name}",
        'START_POINTS': points_layer,
        'STRATEGY': 0,
        'TOLERANCE': 0.0,
        'TRAVEL_COST': 250.0,
        'VALUE_BACKWARD': '',
        'VALUE_BOTH': '',
        'VALUE_FORWARD': ''}
    )