Exporting photos from Survey123 GDB file with custom filenames?

Question

I have a Survey123 Survey that I've been using to collect data with multiple pictures on each datapoint.

Each datapoint has a barcode to correspond with a physical sample taken from each point. I would like to export the photos out of the survey data, which I have done by creating a script in ArcMap, using the code below from this article:

import arcpy
from arcpy import da
import os

inTable = arcpy.GetParameterAsText(0)
fileLocation = arcpy.GetParameterAsText(1)

with da.SearchCursor(inTable, ['DATA', 'ATT_NAME', 'ATTACHMENTID']) as cursor:
    for item in cursor:
        attachment = item[0]
        filename = 'ATT' + str(item[1])
        open(fileLocation + os.sep + filename, 'wb').write(attachment.tobytes())
        del item
        del filename
        del attachment

However this does not allow me to rename the file using the barcode that is associated with that datapoint. So when the photos are exported, they are no longer associated with the relevant data.

I would like to tell the SearchCursor function to also take the sample_id field from the data table and add that to the filename. When I add the sample_id to the array inside the SearchCursor function like this:

with da.SearchCursor(inTable, ['DATA', 'ATT_NAME', 'ATTACHMENTID', 'sample_id']) as cursor

I get the following error: RuntimeError: A column was specified that does not exist.

I would like help in calling the sample_id field into the function properly so it can be added to the filename variable.

Answer 1

If your survey submits data to a feature class called 'Form_2' (a common default name), and the survey includes image questions (photos, signatures, etc), then...

There will be a table of related images called 'Form_2_ATTACH'. This table will have a GUID field called "REL_GLOBALID". This contains the GUID (global ID) of the survey record that the image belongs with. You can use this to join an image with it's related survey data and then you can get all the information you need.

Try something like this (untested!):

import arcpy, os

pathToGDB = arcpy.GetParameterAsText(0)
fileLocation = arcpy.GetParameterAsText(1)

surveyFC = os.path.join(pathToGDB, "Form_2")
imageTable = os.path.join(pathToGDB, "Form_2__ATTACH")
imageSurveyView = "imageSurveyView"  #  Using a variable instead of a string makes typo errors less likely

arcpy.MakeTableView_management (imageTable, imageSurveyView)
arcpy.AddJoin_management (imageSurveyView, "REL_GLOBALID", surveyFC, "GlobalID", "KEEP_ALL")

#  print [f.name for f in arcpy.ListFields(imageSurveyView)]

with arcpy.da.SearchCursor(imageSurveyView, ['Form_2__ATTACH.DATA', 'Form_2__ATTACH.ATT_NAME', 'Form_2__ATTACH.ATTACHMENTID', 'Form_2.sample_id']) as cursor:
    for row in cursor:
        attachment = item[0]
        filename = 'ATT_{}_{}_{}'.format(row[1], row[2], row[3])
        with open(os.path.join(fileLocation, filename), 'wb') as f:
            f.write(attachment.tobytes())

(Note the change to the first parameter - this is the GDB, not the table. You may want to include both as separate parameters. I've hard-coded the table name in the code.)

This will join the image table to the survey feature class, and then the SearchCursor can iterate over the rows of the table view instead of the table itself. You can now use fields from both the photos table and the form table in the SearchCursor fields, including your sample_id field.

NB: You may need to fully qualify your fields in the SearchCursor to include the table name, because the join means there are multiple tables behind this table view. Eg, use Form_2__ATTACH.ATT_NAME instead of ATT_NAME and Form_2.sample_id instead of sample_id.

The above code would include the ATT_NAME, the ATTACHEMENTID and the sample_id in the filename. But it's up to you which of these items you actually want to include (but if you include ONLY the sample_id, you may have several photo files that would end up with the same name and overwrite each other).

(Note that there will also be a 'Form_2__ATTACHREL' which is what ESRI call a relationship class. You can ignore this for these purposes, as you can do your join in code and use that instead. The relationship class does not contain any data (usually). It just defines the relationship between the form data feature class and the images table, including which fields are used as the key for joining the two. It is used by ArcMap or ArcGIS Online to be able to navigate from a parent record to a child record easily.)

Answer 2

I have found some code I was using a Python Toolbox, it was setup for use with Collector for ArcGIS, but I think the same rules will apply to the output from Survey123. I cannot take credit for this code, it is a mix and match of scripts and ideas from this site over the years. I hope it is condensed and applicable to your application.

def execute(self, parameters, messages):
        origTable = parameters[0].ValueAsText #Input Featureclass
               attachTable = parameter

    s[1].ValueAsText # if no attachTable given, append __ATTACH to origTable
           nameField = parameters[3].ValueAsText  # appropriate name field in origTable
           fileLocation = parameters[2].ValueAsText #Output Folder

           origFieldsList = ["GlobalID", "OBJECTID", nameField] # GlobalID for linking, OBJECTID for renaming, nameField for renaming

        # Use list comprehension to build a dictionary from a da SearchCursor
           valueDict = {r[0]:(r[1:]) for r in arcpy.da.SearchCursor(origTable, origFieldsList)}


           count = 1
           with arcpy.da.SearchCursor(attachTable, ['DATA', 'ATT_NAME', 'ATTACHMENTID', 'REL_GLOBALID']) as cursor:
            for item in cursor:
                attachment = item[0] # attachment data


                filenum = "ATT" + str(item[2]) + "_"

                filename = filenum + str(item[1])  # this will be the filename if linking fails

            # store the Join value of the row being updated in a keyValue variable
                keyValue = item[3] # REL_GLOBALID
            # verify that the keyValue is in the Dictionary
                if keyValue in valueDict:
                # transfer the values stored under the keyValue from the dictionary to the updated fields.
                    obID = valueDict[keyValue][0]

                # remove invalid filename characters, replace spaces and periods, limit length
                    check = str(valueDict[keyValue][1])

                    if check == 'None':  #Check for Null Values
                        namefield = "NoValue"
                    else:
                        namefield = re.sub('[^0-9a-zA-Z]+', '_', valueDict[keyValue][1])[:18]



                # Create a unique filename  ObjectID_AttachmentID_namefield.ext
                    ext = filename.rsplit('.', 1)[-1] # keep extension of original file
                    filename = "{}_{}_{}.{}".format(namefield,"OID" + str(obID),"ATTID" +str(item[2]),ext)

                arcpy.AddMessage( "Writing: {}{}{}".format(fileLocation, os.sep, filename))
                arcpy.AddMessage ("Processing: " + str(count))
                count = count +1

                open(fileLocation + os.sep + filename, 'wb').write(attachment.tobytes())
                del item
                del filenum
                del filename
                del attachment
            del count
    return

There are 4 parameters, in order they are:

1. origTable - the input feature class - the point feature from your survey

2. AttachTable - This is the table containing the attachments, it is usually the original table name with "__ATTACH" on the end.

3. nameField - appropriate name field in origTable - the name you want to add to the attachment exported

4. fileLocation - a folder path

You will need to code these into your script. The script then looks over the REL_GLOBAL ID in the Attachment Table and finds the appropriate GLOBAL ID in the main table and looks for the field you specified as an input, and then assigns that name to the attachment name. If there are more than one attachments per point, it adds a number to the end.