1

I have the following script: 

import arcpy
from arcpy import env 
import datetime
import os     


env.workspace = r"Q:\\SDE\Direct Connection to Contours.sde"
dataFC = arcpy.ListFeatureClasses("","","Contours")
outpath = r'C:\Users\TEST\Desktop\SDE Conversion Project\TEST.gdb'
query = "objectid > 0 AND objectid <= 200000 "

for fc in dataFC: 
        if fc == 'Contours':
            output = outpath + os.sep + fc
            arcpy.Select_analysis(fc, output+'200k', query)

Currently it selects records that have objectid between 0 and 200,000 and creates a feature class called Contours 200k in the TEST gdb.

What I want to be able to do is make a multiprocessing script that utilises one core to calculated between object id 0 and 200k, another that calculates betweek 200,001 and 400k and so on until the whole feature class is copied over as separate feature sets to the TEST gdb.

How would I go about doing that? I'm trying to do this since the bottleneck of copying from the sde is the network connection and testing with separate files where I had the ranges set manually improved the overall speed a lot.

Improve this question
3
  • 1
    ArcGIS desktop is a single thread application, you can use subprocess.Popen to call multiple scripts but you will run into a self locking situation as only one process can write to a geodatabase at a time, you could create a GDB in your os.environ.get('TEMP') for each process and write there then use append or merge to join the multiple databases together though I would think that this would be no quicker in the long run... select by location on the other hand could be split up like this to gain better performance.
    – Michael Stimson
    Apr 1, 2019 at 3:51
  • I wrote a short blog providing a template over on geonet, suggest you read that to understand how to implement multiprocessing in arcpy. There are also many other related threads on geonet you should explore.
    – Hornbydd
    Apr 1, 2019 at 10:24
  • This has the feel of an XY Problem. The best way to improve contour line performance is to intersect with a coarse fishnet and dissolve on fishnet_fid (or FID plus elevation). This reduces the extent of individual features to the point that the spatial index is useful and reduces the feature count so that it is performant. Multi-threading is not generally a solution to a networking bottleneck.
    – Vince
    Apr 1, 2019 at 10:27

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.