2

So I'm running a simple line FC through this:

# geh = inputFC
# fields = ["list","of","fieldnames"]
def clause(fields,br_cl_val):
    clause = """{0} = {1}""".format(arcpy.AddFieldDelimiters(geh,fields),br_cl_val)
    return clause
arcpy.MakeFeatureLayer_management(geh,"to_diss",clause(fields[1],0))
arcpy.Dissolve_management("to_diss", "geh_pp_k0_D", "", "Shape_Leng SUM", "MULTI_PART", "UNSPLIT_LINES")

This code runs fine with my testing dataset of ~3000 features which is a direct subset of the actual production dataset I write this code for. However the latter contains ~180000 features and fails with:

ERROR 000369:Invalid input field(s)

Boths datasets have the same CRS, same fieldmapping, (even trying from the) same GDB. What could be the problem here?

(10.2.2; 64bit processing)

PS: Is there at least a way to verify wether the creation of the featurelayer I dissolve on was successfull done with the production dataset?

  • Try it with background geoprocessing turned off. – Michael Stimson Nov 16 '17 at 4:50
  • I'll try but the rest of the script exceeds the 2gb ram barrier. – maxwhere Nov 16 '17 at 5:00
  • 1
    It's 4 GiB that is the maximum addressable by a 32bit process. Esri tools shouldn't use that much memory, they're descendants of process that I ran back in the workstation days with 8 MB of memory, the tools should internally 'dice' the inputs to conserve RAM, the tradeoff is that they could be slower; but still, slower in 32bit compared to not working at all in 64bit I know which I'd choose. However most tools still use temporary shape files that have a maximum size of 2 GiB regardless of 32 or 64 bit so you might need to do them in sections for very large dissolves from a file/enterprise db. – Michael Stimson Nov 16 '17 at 5:08
  • 1
    You could try it with option single part and then dissolve again with multi part (if multipart is what you want in the end) – BERA Nov 16 '17 at 7:16
  • So after tinkering around with the suggested solutions i find that the results gotta be convenient enough when I'm using singlepart and I hope that tolerance still applies to my other production data. Still strange error though. Anyway: Thank you very much for your inputs! – maxwhere Nov 20 '17 at 6:38
1

Sounds like an issue with memory and/or possibly the complexity of your data, you don't actually describe what it is you are dissolving. Simple polygons, or insanely complex habitat maps?

Since the release of ArcGIS Pro, when I have had issues crunching large datasets in ArcMap I have run exactly the same tool in ArcGIS Pro and it processes it. I would suggest you try it in that software if you are finding the python scripts failing in ArcMap.

  • Allthough I can't falsify your claim because I can't just change from one arcgis version to another like i change my underwear your first line gives me the impression that you didn't actually read my question and just came here to say how efficient you're "crunching large datasets [..] in ArcGIS Pro and it processes it". – maxwhere Nov 20 '17 at 7:12
  • 1
    You are right I missed that it was a line dataset you are talking about, may be some highlighting formatting would have helped? But my comment still holds true as what is it your lines represent, simple roads with a few vertices or complex highly tortuous lines, your are not describing your dataset. We don't even know from your question what data source your data is (i.e. is it a shapefile/personal geodatabase). As for ArcGIS Pro, I guess you are not using it yet as you would know there is nothing to change you add data as you do in ArcMap (unless it is a personal geodatabase). – Hornbydd Nov 20 '17 at 12:27
  • The dataset is basically the output of the central line function used on a single sidewalk polygon of a whole city and therefore very tortuous. Maybe you are right on the lack of formatting in my post but surely you're not on its concise nature because as seen on several other posts here 'simple line FC' and having it 'even tried from the same GDB ' implies that a file geodatabase is beeing used doesn't it... – maxwhere Nov 20 '17 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.