0

I have the following script:

        workspace = r"D:\\Python\\Cend.gdb"
        MEGIS_ELECTRIC_CircuitSource = "CircuitSource"
        Fc = "ElectricDistribution\\PriOHConductor"
        Fc2 = "ElectricDistribution\\PriUGConductor"
        
        arcpy.env.workspace = workspace
        arcpy.env.overwriteOutput = True 

        ELECTRIC_CircuitSource_OBJID =[]
        ELECTRIC_CircuitSource_ID =[]
        ELECTRIC_CircuitSource_Color =[]
        FldLst = ['objectid','ID','Color']

        print("Started")
        print("Processing for...."+str(Fc))
        with arcpy.da.SearchCursor(MEGIS_ELECTRIC_CircuitSource, FldLst) as cursor:
            for row in cursor:
                ELECTRIC_CircuitSource_OBJID.append(row[0])
                ELECTRIC_CircuitSource_ID.append(row[1])
                ELECTRIC_CircuitSource_Color.append(row[2])

        Fields=["objectid","FeederID","Rend"]
        
        PriOHConductor_objid =[]
        PriOHConductor_fdrid =[]
        PriOHConductor_rend =[]
        
        with arcpy.da.SearchCursor(Fc, Fields) as cursor:
            for row in cursor:
                PriOHConductor_objid.append(row[0])
                PriOHConductor_fdrid.append(row[1])
                PriOHConductor_rend.append(row[2])
        
        PriOHConductor_objid_upd =[]
        PriOHConductor_rend_upd =[]
        my_dict = {}
        for a,b,c in zip(PriOHConductor_objid,PriOHConductor_fdrid,PriOHConductor_rend):
            for x,y,z in zip(ELECTRIC_CircuitSource_OBJID,ELECTRIC_CircuitSource_ID,ELECTRIC_CircuitSource_Color):
                if str(b).lower() == str(y).lower() :
                    if str(c).lower() != str(z).lower() :
                        if a not in my_dict:
                            my_dict[a] = z
        print("Match Count----"+str(len(my_dict)))
        Fields =["objectid","Rend"]
        field = "Rend"
        keys_iter = iter(my_dict.keys())
        batch_size = 50000
        total_batches = math.ceil(len(my_dict) / batch_size)
        batch_cnt = 1
        while True:
            batch_keys = list(itertools.islice(keys_iter, 50000))
            if not batch_keys:
                break
            batch_dict = {k: my_dict[k] for k in batch_keys}
            print("Batch :::: "+str(batch_cnt)+"/"+str(total_batches+1))
            batch_cnt = batch_cnt + 1
            qur = "objectid in("
            for i in batch_dict.keys():
                qur = qur+str(i)+","
            qur = qur+")"
            qur = qur.replace(",)",")")
            with arcpy.da.Editor(workspace) as edit:   
                with arcpy.da.UpdateCursor(Fc, Fields,qur) as cursor:
                    for row in cursor:
                        row_objid = str(row[0])
                        for objid,clr in zip(PriOHConductor_objid_upd,PriOHConductor_rend_upd):
                            if row_objid == objid:
                                row[1] = str(clr)
                        cursor.updateRow(row)

This code is working perfectly on file geodatabase and giving me output:

Started
Processing for....ElectricDistribution\PriOHConductor
Match Count----877169
Batch :::: 1/18.0
Batch :::: 2/18.0
Batch :::: 3/18.0
Batch :::: 4/18.0
Batch :::: 5/18.0
Batch :::: 6/18.0
Batch :::: 7/18.0
Batch :::: 8/18.0
Batch :::: 9/18.0
Batch :::: 10/18.0
Batch :::: 11/18.0
Batch :::: 12/18.0
Batch :::: 13/18.0
Batch :::: 14/18.0
Batch :::: 15/18.0
Batch :::: 16/18.0
Batch :::: 17/18.0
Batch :::: 18/18.0

Done
Elapsed time: 0 days, 0 hours, 9 minutes, 50 seconds

However while running on a an enterprise geodatabase (SQL Server) following error Occurs:

Started
Processing for....MEGIS.ELECTRIC.ElectricDistribution\MEGIS.ELECTRIC.PriOHConductor
Match Count----877169
Batch :::: 1/18.0
('Error:', RuntimeError('Underlying DBMS error [[Microsoft][ODBC Driver 17 for SQL Server]
[SQL Server]The query processor ran out of internal resources and could not produce a query plan.
This is a rare event and only expected for extremely complex queries or queries 
that reference a very large number of tables or partitions.
Please simplify the query.
If you believe you have received this message in error, contact Customer Support Services for more information.]
[MEGIS.ELECTRIC.PriOHConductor]',))
Elapsed time: 0 days, 0 hours, 5 minutes, 49 seconds

I am scratching my head around it.

1
  • 1
    This is a Microsoft error, asking you to contact Microsoft. Did you? What about Esri Tech Support?
    – Vince
    Commented Apr 7, 2023 at 11:39

1 Answer 1

0

After careful analysis , It turned out that the server cannot handle the query for 50K records, I changed the parameter to 10K records( batch_size = 10000) and was able to process it.

Thanks for the help and support.

Not the answer you're looking for? Browse other questions tagged or ask your own question.