1

Referring to this topic How to randomly subset X% of selected points?. I tried in ArcPy and I can't figure out how to change this code to randomly select 50% of the polygons based on the attribute (CLASS) So that each set will have equal attributes (ex. 50% of forests, water, meadow in one set and 50% of forests, water, meadow in the other set).

def SelectRandomByPercent (layer, percent):
#layer variable is the layer name in TOC
#percent is percent as whole number  (0-100)
if percent > 100:
    print "percent is greater than 100"
    return
if percent < 0:
    print "percent is less than zero"
    return
import random
fc = arcpy.Describe (layer).catalogPath
featureCount = float (arcpy.GetCount_management (fc).getOutput (0))
count = int (featureCount * float (percent) / float (100))
if not count:
    arcpy.SelectLayerByAttribute_management (layer, "CLEAR_SELECTION")
    return
oids = [oid for oid, in arcpy.da.SearchCursor (fc, "OID@")]
oidFldName = arcpy.Describe (layer).OIDFieldName
path = arcpy.Describe (layer).path
delimOidFld = arcpy.AddFieldDelimiters (path, oidFldName)
randOids = random.sample (oids, count)
oidsStr = ", ".join (map (str, randOids))
sql = "{0} IN ({1})".format (delimOidFld, oidsStr)
  • Welcome to GIS SE! We're a little different from other sites. We're a Q&A site, not a discussion forum. For questions that involve code we ask that you show us where you are stuck with your own code by including a code snippet in your question. There is an edit button beneath your question which will enable you to do that and a {} button that enables you to format any highlighted code nicely. Please check out our short tour for more about how the site works. Thanks. – PolyGeo Nov 25 '19 at 11:47
1

That code is a good start. Before I go about making recommendations you must be made aware that getting an exact percent is very difficult, the sample to be selected from may not be a multiple of the percent, for example 50% of 3 is 1.5 but you can't have half a feature so it becomes either 1 (int/floor) or 2 (ceil).

In order to use any kind of unique field you must pass it to the function, so it is necessary to change the def SelectRandomByPercent (layer, percent): to def SelectRandomByPercent (layer, percent,uField):; global variables do exist in python but I would discourage you from pursuing that route.

Every person is different so it is reasonable to say that individuals code differently, I see a lot of unnecessary lines in the code sample where others would see coding for understanding the processes, to this end I started from a blank slate:

def SelectRandomByPercent (layer, percent,uField):
        uFieldField = arcpy.ListFields(layer,uField)[0] # get the input layer field as a field object
        # Validate the input field type, only some field types are valid
        # for the purposes of this script: either simple numeric or string
        if uFieldField.type.lower() == 'string':
            uFieldString = True    # field is a string
        elif uFieldField.type.lower() in ['double', 'integer', 'single', 'smallinteger']:
            uFieldString = False   # field is a valid numeric type
        else:
            # field is Blob, Date, GUID, Raster etc.. not a valid field for this script
            arcpy.AddError('indicated field {} is not a valid field type'.format(uField))
            return

        # I want the percent to be between 0 and 1 for later calculation
        # as I'm using len(list)*percent, alternately you could leave the
        # percent as between 0 and 100 and change to len(list)/percent
        if percent > 1:
                percent = float(percent) # float the int first otherwise the result is 0
                percent /= 100
                if percent > 1:
                        print "percent is greater than 100"
                        return
        if percent < 0:
                        print "percent is less than zero"
                        return

        # get a list of all the unique values in the uField
        # and make them into a list to be iterated later
        uVals = []
        with arcpy.da.SearchCursor(layer,uField) as SCur:
                for SRow in SCur:
                        if not SRow[0] in uVals: # consider using upper or lower case here
                                uVals.append(SRow[0])

        selectOIDs = []         # the list of 'final' selection OIDs, starting with empty
        for this_uVal in uVals: # for each unique value in the uField
                # get all the ids associated with this unique value
                if uFieldString:
                    # using a quoted where clause for a string field type
                    with arcpy.da.SearchCursor(layer,'OID@','{} = \'{}\''.format(uField,this_uVal)) as SCur:
                            defQ_IDS = []                # the OIDs for just this unique value, starting with empty
                            for SRow in SCur:
                                    defQ_IDS.append(SRow[0]) # add each OID in turn to the list, I don't use += because SRow[0] isn't a list

                            # Now add a subset to the selection OIDs if you want to round up then import math
                            # and use int(ceil(len(defQ_IDS)*percent)) either way the sample must be an integer.
                            # There is no need to get a count of features as the OIDs are loaded into the list
                            # just get the length of the list and that will be the number of features
                            selectOIDs += random.sample(defQ_IDS,int(len(defQ_IDS)*percent))
                else:
                    # using an unquoted where clause for numeric types
                    with arcpy.da.SearchCursor(layer,'OID@','{} = {}'.format(uField,this_uVal)) as SCur:
                            defQ_IDS = []                # the OIDs for just this unique value, starting with empty
                            for SRow in SCur:
                                    defQ_IDS.append(SRow[0]) # add each OID in turn to the list, I don't use += because SRow[0] isn't a list

                            # Now add a subset to the selection OIDs if you want to round up then import math
                            # and use int(ceil(len(defQ_IDS)*percent)) either way the sample must be an integer.
                            # There is no need to get a count of features as the OIDs are loaded into the list
                            # just get the length of the list and that will be the number of features
                            selectOIDs += random.sample(defQ_IDS,int(len(defQ_IDS)*percent))


        # now that we have a percent of each unique values of uField actually
        # do the selecting against the layer, but first get the OID field name
        Des      = arcpy.Describe(layer)
        OIDfield = Des.oidFieldName
        arcpy.SelectLayerByAttribute_management(layer,'NEW_SELECTION','{} in ({})'.format(OIDfield, ','.join([str(sID) for sID in selectOIDs])))

This works like the example code but has a few minor differences.. Every feature is iterated twice: the first to get the unique value of the given field and secondly to get the list of OIDs that relate to each unique code, from which a random sample of OIDs are added using the += operator for the list, explained on Stack Overflow. Don't forget that you need to import random at the top of your script to use this lib in this function.

  • 1
    Although the code provided should work I haven't actually tested it, if there's an error it's probably an easy fix of a typo provided I can get the line number to limit the search. Error 358 links to gis.stackexchange.com/questions/61499/… which leads me to the last line where I can see the problem, it should be ','.join([str(sID) for sID in selectOIDs]) to turn the list of numbers into a comma delimited string to insert into the SQL selection, I'll fix that up now. Try the updated code now, it should work. – Michael Stimson Nov 26 '19 at 23:03
  • 1
    I have tested the code block, recopied from this post, and it works for me. Try copying it again, or the last line at least as the error on the last line which should be fixed as it is the only instance of SelectLayerByAttribute; while I was testing the script I added some extra lines to handle numeric field types and reject invalid ones which makes this code more robust but a bit longer, for the purposes of the question though the pre-edit code should suffice with the modified last line. – Michael Stimson Nov 27 '19 at 22:40
  • 1
    That's interesting, what data type is your data? I tested it on a shapefile feature layer. Are you passing a layer to the function or a feature class path? (yes, that makes a difference but can be resolved). Is this being run in CMD, ArcMap or ArcCatalog? Does your layer currently have a selection before being passed? When a layer has a selection only the selected features are passed to the script, if you run the script twice at 50% you will end up with 25% selection. Is your layer valid? (no red exclamation mark next to it) - again, that can be coded for but adds to an already big script. – Michael Stimson Nov 28 '19 at 22:35
  • 1
    The only change that I made was to float the percent... that's pythons' duck-typing en.wikipedia.org/wiki/Duck_typing. If you enter your percent as 50 it's an integer so the /= comes back as 0, I was entering 0.5 which is a float and didn't need to be divided by 100 so was working as expected, the fix is to implicitly float(percent) before /= which makes the result for 50 50.0 then 0.5 in turn and the code works as expected. – Michael Stimson Dec 1 '19 at 23:29
  • 1
    Pythons' duck typing can be a big problem for coding as you're never absolutely certain of the type of a variable unless you've typed it stackoverflow.com/questions/402504/…. – Michael Stimson Dec 1 '19 at 23:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.