0

There is a Python script intended to create a UPDM geodatabase, it receives several parameters, among them, database admin user, database dataowner user, and their passwords.

The script runs using the ArcGIS Pro 2.8 Python, it starts its process, creates the database, as well as connection files (.sde). In some point of the script, it has been created different structures: tables, relationships, domains, etc. but then it fails while calling:

arcpy.da.ListDomains(self.database)

Here you can see the function:

def create_domains(self, domains=None):
    # Function to create domains within the geodatabase based on the JSON file
    if domains is None:
        domains = self.parser.parse_key(self.json_config, self.DOMAINS)
    for dm in domains:
        d_name = self.parser.parse_key(dm, self._NAME)
        d_desc = self.parser.parse_key(dm, self._DESCRIPTION)
        d_data_type = self.parser.parse_key(dm, self.DATA_TYPE)
        d_type = self.parser.parse_key(dm, self.DOMAIN_TYPE)
        d_values = self.parser.parse_key(dm, self.DOMAIN_VALUES)
        existing_domains = arcpy.da.ListDomains(self.database)
        domain_exist = any(check_domain_exists(d_name, d) for d in existing_domains)
        if domain_exist:
            self.logger.add("Domain {0} already exists".format(d_name))
            continue
        arcpy.CreateDomain_management(
            self.database,
            d_name,
            d_desc,
            d_data_type,
            d_type)
        self.logger.add("Domain {0} is created".format(d_name))
        for c_v in d_values:
            code = self.parser.parse_key(c_v, self._CODE)
            value = self.parser.parse_key(c_v, self.VALUE)
            arcpy.AddCodedValueToDomain_management(
                self.database,
                d_name,
                code,
                value)
        self.logger.add("Values for domain {0} is created".format(d_name))
        if d_type == "RANGE":
            d_max = self.parser.parse_key(dm, "MaxValue")
            d_min = self.parser.parse_key(dm, "MinValue")
            arcpy.SetValueForRangeDomain_management(self.database, d_name, d_min, d_max)

The error is quite weird (at least for me):

ERROR 1: Can't load requested DLL: C:\ArcGISPro\bin\gdalplugins\gdal_MG4Lidar.dll
1114: A dynamic link library (DLL) initialization routine failed.

ERROR 1: Can't load requested DLL: C:\ArcGISPro\bin\gdalplugins\gdal_MG4Lidar.dll
1114: A dynamic link library (DLL) initialization routine failed.

ERROR 1: Can't load requested DLL: C:\ArcGISPro\bin\gdalplugins\gdal_MG4Lidar.dll
1114: A dynamic link library (DLL) initialization routine failed.

ERROR 1: Can't load requested DLL: C:\ArcGISPro\bin\gdalplugins\gdal_MG4Lidar.dll
1114: A dynamic link library (DLL) initialization routine failed.

ERROR 1: Can't load requested DLL: C:\ArcGISPro\bin\gdalplugins\gdal_MG4Lidar.dll
1114: A dynamic link library (DLL) initialization routine failed.

11:51.29.583441 ERROR C:\temp\Installation\Data\DataOwner.sde
11:51.29.589433 ERROR ['Traceback (most recent call last):\n', '  File ".\\Arcpy\\InstallDataModel.py", line 95, in main\n', '  File "C:\\temp\\Installation\\Arcpy\\Classes\\JSONGeoDataModelClass.py", line 598, in create_domains\n    existing_domains = arcpy.da.ListDomains(self.database)\n', 'OSError: C:\\temp\\Installation\\Data\\DataOwner.sde\n']

Furthermore, the ArcGIS Pro Crash Window appear 4 times (probably one time for each crash in the request of the gdal_MG4Lidar.dll).

In this point of the execution, this line has been called several times, the script has been using the En terprise geodatabase connection file for several actions, inside the script there is no direct call to GDAL libraries (I assume it is an internal call from the ArcGIS Pro libraries/modules) and despite the message, I can verify the existence of the connection file and even use it in a new ArcGIS Pro instance.

It would be expected that domains were created and no error appears.

UPDATE: After digging a little bit in the behavior of the memory in the computer while the script run, then I implemented the use of the garbage collector at the end of the functions, this does not remove the error.

I decided to split the execution in several parts, each part called independently. The script is still failing, but now, the error is :

CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!
CPLGetTLSList(): TlsAlloc() failed!

According t this GDAL github page the error now is generated in that point, but the reason is still unknown.

2
  • What is the minimal amount of code, that calls your function,and which can be run to reproduce your problem?
    – PolyGeo
    Oct 11, 2021 at 11:53
  • Unfortunately I do not have a minimal amount of code to reproduce the behavior, I already tried to isolate the behavior, but every attempt leads me to a success, this behavior only happens when I execute the whole script. That is why I tried to explain all the different actions made by the script. As I did not find any other post with the same errors (code error), that is why i created the question.
    – XtianGIS
    Oct 11, 2021 at 12:19

1 Answer 1

1

After digging in the behavior of the computer, and with the latest error messages I think, the issue is related to the thread management implemented underneath by arcpy library.

I split the script in several parts, and call every part from a new python process, in this way the number of threads does not exceed the allowed amount of threads. Example:

python.exe script_part1.py
python.exe script_part2.py
python.exe script_part3.py
python.exe script_part4.py
1
  • Not python sub-process. a whole new invocation for python for each part of of the script
    – XtianGIS
    Oct 19, 2021 at 6:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.