I'm writing lot's of features to a FileGDB using OGR.

The slow point in my script is checking that the feature isn't already in the Feature Class, based on the ID. Currently I'm using a function and:

crfLayer.SetAttributeFilter("ID = " + str(crfID))
Count = crfLayer.GetFeatureCount()
if Count == 0:
    return 0
    return 1

I've tried looping creating a list from the file and checking if the ID is in it, however this is slower.

Does anybody have any alternatives to the code above?

  • Did you try to add an attribute index? – Matthias Kuhn Jul 18 '13 at 6:32
  • Does that work even with File Geodatabases? The link provided is for a shapefile. – user19466 Jul 18 '13 at 7:14

Adding an attribute index on the ID field to your geodatabase would help but is not available in the desktop Basic version of ArcGIS (so check you licence). However, I'm not sure how effective that would be if you are doing everything from OGR, so here's an idea:

Try using a list comprehension instead of a loop to create your list of IDs. Python is heavily optimized for list comprehensions and this normally yields a significant speed boost over creating a list iteratively. So create your list something like this:

result = foo.ExecuteSQL("SELECT DISTINCT id FROM bar")
myList = [result.GetFeature(i).GetField(0) for i in range(result.GetFeatureCount())]

In conjunction with this, what you could do is make a second list of the ids from your features to be added (with another list comprehension) and then get the difference of the two lists like this:

featuresToAdd = list(set(listOfAddableFeatures) - set(listOfExistingFeatures))

You now have a list of only the ids you're interested in adding. You can make an OGR selection set based on this list and then union it with your existing features set (thus avoiding adding features individually). Again, I would look to make the selection set in one go rather than iteratively

sqlString = "SELECT * FROM myTable WHERE id IN " + str(tuple(featuresToAdd))
resultToUnion = addUs.ExecuteSQL(sqlString)
  • Thanks. I want to try and stay clear of using ESRI for this process as it's five times slower. I can see what you are trying to do with the solution but the myList line turns up the error: AttributeError: 'NoneType' object has no attribute 'GetField' – user19466 Jul 18 '13 at 7:57
  • This suggests that there is a problem with your ExecuteSQL command (i.e. it is returning a null object and has therefore failed). 'foo' will be your datasource (from ogr.Open()) and 'bar' will be your featureclass name. I just tested it and it works for me (make sure your field name is correct) – MappaGnosis Jul 18 '13 at 10:04
  • Ok, thanks for explaining, I've assigned foo and bar correctly, the result line returns something as result.GetFeatureCount() returns the number of features in the feature class. removing the list comprehension and putting result.GetFeature(i) in a for loop returns a SWIG object. When I try to do anything further I get the NoneType error. – user19466 Jul 19 '13 at 0:15

layer.GetLayerDefn().GetFieldIndex("field_name") >= 0

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.