1

Im iterating through folders and subfolders to find shapefiles (~140 shapefiles, ~20 GB total) then applying fixgeometries and importintopostgis to import them to a postgis database.

While code is executing I would like QGIS to print out for example Processing shapefile123.shp but QGIS is freezing while code is running. All the while the import into postgis is working, I keep track on the side using pgadmin.

What is causing QGIS to freeze while the code is executing and how can I fix it? CPU and mem usage is about 50 %. Maybe Tasks is what I want?

enter image description here

from subprocess import call
import os, datetime

a = datetime.datetime.now()
print ('start',str(datetime.datetime.now().time()))

#Create database
call(["psql", "-U", "postgres", "-c" "drop database if exists lmv;"])
call(["psql", "-U", "postgres", "-c" "create database lmv;"])
call(["psql", "-U", "postgres", "-d", "lmv", "-c", 'create extension postgis;create extension tablefunc;'])

root = r'C:\GIS\data' #Folder with subfolders with shapefiles

for path, subdirs, files in os.walk(root):
    for name in files:
        if name.endswith('.shp'):
            #print(os.path.join(path,name))
            if 'ok_riks' in path:
                prefix = 'ok_'
            elif 'vk_riks' in path:
                prefix = 'vk_'
            elif 'tk_riks' in path:
                prefix = 'tk_'
            elif 'fj_riks' in path:
                prefix = 'fj_'
            else:
                prefix = 'unknownsrc_'
            print('Reparerar: ', prefix+name)
            fixgeom = processing.run("native:fixgeometries", 
                {'INPUT':os.path.join(path,name),'OUTPUT':'TEMPORARY_OUTPUT'})['OUTPUT']
            print('Importerar: ', prefix+name)
            print('Tabellnamn: {0}'.format(prefix+name.replace(".shp","")))
            processing.run("qgis:importintopostgis", 
                {'INPUT':fixgeom,'DATABASE':'lmv','SCHEMA':'public','TABLENAME':prefix+name.replace(".shp",""),
                'PRIMARY_KEY':'','GEOMETRY_COLUMN':'wkb_geometry','ENCODING':'UTF-8','OVERWRITE':True,
                'CREATEINDEX':True,'LOWERCASE_NAMES':True,'DROP_STRING_LENGTH':False,'FORCE_SINGLEPART':False})

print ('stop',str(datetime.datetime.now().time()))
b = datetime.datetime.now()
print('Total tid {0} s ({1} min)'.format((b-a).seconds, ((b-a).seconds/60)))

Four hours later it unfreezes:

enter image description here

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.