1

I'm getting strange behavior from ArcGIS Pro 2.9.4 / ArcPy 2.9.

In a nutshell, I have an ArcPy script that exports PDF maps correctly if I run it in Python Window, but omits one specific layer when run as a Python Toolbox tool.

I have a shapefile containing occupancy data (1 or 0) for multiple species, one field per species. I wrote a script that iterates through and copies data for one species into another field for display (CalculateField), then updates the map title text, exports the layout as a PDF, and then repeats it for the next species... If I run it in the Python Window, no problem, it works fine. I get a set of species map PDFs.

My problem started when I adapted my script to create a Python Toolbox tool. The tool apparently works with no error messages. It iterates through, updates the data field, updates the title, exports the maps - EXCEPT, every PDF omits the shapefile with the occupancy data! I cannot for the life of me figure out why it is omitting only the occupancy shapefile. The occupancy data displays on my monitor just fine - it just never makes it into the PDF.

Things I've tried:

  • Removed all other map layers to make sure nothing is covering it.
  • Added code to turn the occupancy layer off and on again to try and trigger a redraw prior to exporting the PDF.
  • Checked the shapefile to make sure data is being copied correctly. It is.
  • No joins or relates are in the shapefile.
  • Ran the code using another layout with another shapefile for another sample location (same problem - it produces map PDFs without the data point layer).
  • Played around with PDF resolution, changed from exporting LAYERS_ONLY to LAYERS_AND_ATTRIBUTES.
  • Copied the code back out of the Python Toolbox to run in Python Window - it produced the map correctly (points showed) if I use the Python Window.

I'm at a loss. It seems like maybe it's simply a bug?

I'm attaching the code for the Python Tool and the code that works fine in Python Window.

Code for the Python Tool

# -*- coding: utf-8 -*-

import arcpy, os


class Toolbox(object):
    def __init__(self):
        """Define the toolbox (the name of the toolbox is the name of the .pyt file)."""
        self.label = "Wildlife Map Production"
        self.alias = "mapproduction"

        # List of tool classes associated with this toolbox
        self.tools = [OccupancyMap]


class OccupancyMap(object):
    def __init__(self):
        """Define the tool (tool name is the name of the class)."""
        self.label = "Occupancy Map Production"
        self.description = "This tool produces a set of wildlife occupancy maps and exports them as PDF files to the 'maps' directory in the appropriate report draft folder."
        self.canRunInBackground = False

    def getParameterInfo(self):
        """Define parameter definitions"""
        
        param0 = arcpy.Parameter(
            displayName="Park Code",
            name="parkcode",
            datatype="GPString",
            parameterType="Required",
            direction="Input")

        # Set a value list
        param0.filter.type = "ValueList"
        param0.filter.list = ['CAHA', 'CALO', 'CANA', 'CHAT', 'CONG', 'CUIS', 'FOFR', 'FOMA', 'FOPU', 'FOSU', 'HOBE', 'KEMO', 'MOCR', 'OCMU', 'TIMU']

        param1 = arcpy.Parameter(
            displayName="Sampling Year (4-digit)",
            name="year",
            datatype="GPString",
            parameterType="Required",
            direction="Input")
        param1.filter.type = 'ValueList'
        param1.filter.list = ['2012', '2014', '2015', '2016', '2017', '2020', '2021', '2022', '2023']
        
        
        param2 = arcpy.Parameter(
            displayName="Taxa",
            name="taxa",
            datatype="GPString",
            parameterType="Required",
            direction="Input")
        param2.filter.type = 'ValueList'
        param2.filter.list = ['Anuran', 'Landbird']


        params = [param0, param1, param2]
        return params


    def execute(self, params, messages):
        
        # Set path to current AGP project
        p = arcpy.mp.ArcGISProject("CURRENT")
        
        ### 0.1 INPUT USER-DEFINED PARAMETERS
        parkcode = params[0].valueAsText
        year     = params[1].valueAsText
        taxa     = params[2].valueAsText
        
        ### List of detected species.  
        ### Formatted as alphacode, (NOSPACE), common name, (SPACE) scientific name with parentheses and italics tags.
        PARKLIST = "R://GIS//projects//Wildlife//Occupancy//"+parkcode+"//"+taxa+"_EB_"+year+"_"+parkcode+"_PARKLIST.txt"
        with open(PARKLIST, 'r') as file:
            PARKLIST2 = file.read().replace('\n', '')
        PARKLIST3 = PARKLIST2.replace('"','')   
        PARKSPECIES = PARKLIST3.split(", ")
        
        ### 0.2 SET OUTPUT FOLDER AND INPUT FEATURE
        ###
        
        ### Define output folder (automatic based on year and parkcode).
        ###
        outputfolder = "C://Users//myname//Documents//REPORTS//Expanded Briefs//Drafts//"+taxa+"//"+year+"_"+parkcode+"//maps//"
        
        ### Define input feature shapefile name (automatic based on year, parkcode, and taxa).
        ### e.g., Landbird_EB_2020_CHAT_OccupancyB.shp
        ###
        inFeatures = taxa+"_EB_"+year+"_"+parkcode+"_OccupancyB"  
        
        
        ###
        ### 1.0 ITERATE THROUGH SPECIES AND PRODUCE OCCUPANCY MAPS
        ###
        
        
        #Establish Counter.
        counter1 = 101
        counter2 = str(counter1)
        
        for SPP in PARKSPECIES:
            ### 1.1 Slice out the alphacode from the PARKSPECIES list.
            ###
            string = SPP
            if taxa == "Landbird":
                n = 4
                CURRENTLAYOUT = "Landbird_EB_"+year+"_"+parkcode+"-FigA"
            elif taxa == "Anuran":
                n = 5
                CURRENTLAYOUT = "Anuran_EB_"+year+"_"+parkcode+"-FigA"
        
            alphacode = string[0:n]
            sppname = string[n:]
        
            ### 1.2 Define PDF export parameters.
            ###
            out_pdf = outputfolder+taxa+"_EB_"+year+"_"+parkcode+"_"+counter2+"_"+alphacode+".pdf"
            resolution = 600
            image_quality = "BEST"
            compress_vector_graphics = True
            image_compression = "JPEG"
            embed_fonts = True
            #layers_attributes = "LAYERS_ONLY"
            layers_attributes = "LAYERS_AND_ATTRIBUTES"
            georef_info = False
            jpeg_compression_quality = 100
            clip_to_elements = False
            output_as_image = False
            embed_color_profile = True
            pdf_accessibility = False
            
            ### 1.3 Define Occupancy as field to be updated.
            ###
            FieldName = "Occupancy"
        
            ### 1.4 Execute CalculateField to update Occupancy field with data for current species.
            ###
            expression = "!"+alphacode+"!"
            arcpy.CalculateField_management(inFeatures, FieldName, expression, "PYTHON")
            
        
            ### 1.5 UPDATE LAYOUT to display current species name and nomenclature in title bar.
            ### - Iterate through all layouts.
            ### - Iterate through all text elements in each layout.
            ### - If a text element is called "Species" then rename it to the sppname string.
            ###
            for lyt in p.listLayouts(CURRENTLAYOUT):
                for elm in lyt.listElements("text_element"):
                    if elm.text == "Species":
                        elm.text = sppname
                        elm.visible = False
                        elm.visible = True
        
            ### 1.6 Force a Refresh of the Map
            TYPOB = taxa+"_EB_"+year+"_"+parkcode+"_OccupancyB"
            TYPFA = taxa+"_EB_"+year+"_"+parkcode+"-FigA"
            m = p.listMaps(TYPFA)[0]
            lyrList = m.listLayers()
            for lyr in lyrList:
                if lyr.name == TYPOB:
                    lyr.visible = False
                    lyr.visible = True
            
            
            ### 1.7 Export current map layout as PDF.
            
            Layout = p.activeView
            Layout.exportToPDF (out_pdf, resolution, image_quality, compress_vector_graphics, image_compression, embed_fonts, layers_attributes, georef_info, jpeg_compression_quality, clip_to_elements, output_as_image, embed_color_profile, pdf_accessibility)
        
            ### 1.8 RESET LAYOUT to display the word "Species" in title bar.
            ### - Iterate through all layouts.
            ### - Iterate through all text elements in each layout.
            ### - If a text element is sppname then rename it to "Species".
            for lyt in p.listLayouts(CURRENTLAYOUT):
                for elm in lyt.listElements("text_element"):
                    if elm.text == sppname:
                        elm.text = "Species"
        
            ### 1.9 Update counter by adding 1.
            ###
            counter1 = counter1 + 1
            counter2 = str(counter1)
        
        ### 2.0 Program Termination
        ###
        counter1 = 0
        print ("***  Program Complete   ***")
        print ("*** "+taxa+"_EB_"+year+"_"+parkcode+" ***")
        
        return

Code snippet that works in Python Window:

taxa = "Anuran"
year = "2017"
parkcode = "CONG"
outputfolder = "C://Users//myname//Documents//REPORTS//Expanded Briefs//Drafts//"+taxa+"//"+year+"_"+parkcode+"//maps//"
alphacode = "AAAA"
counter2 = "999"

out_pdf = outputfolder+taxa+"_EB_"+year+"_"+parkcode+"_"+counter2+"_"+alphacode+".pdf"
resolution = 600
image_quality = "BEST"
compress_vector_graphics = True
image_compression = "JPEG"
embed_fonts = True
#layers_attributes = "LAYERS_ONLY"
layers_attributes = "LAYERS_AND_ATTRIBUTES"
georef_info = False
jpeg_compression_quality = 100
clip_to_elements = False
output_as_image = False
embed_color_profile = True
pdf_accessibility = False

Layout = p.activeView
Layout.exportToPDF (out_pdf, resolution, image_quality, compress_vector_graphics, image_compression, embed_fonts, layers_attributes, georef_info, jpeg_compression_quality, clip_to_elements, output_as_image, embed_color_profile, pdf_accessibility)

Here's ArcGIS Pro on the left and the resulting PDF on the right.

enter image description here


I've done some additional tinkering and reduced the code to the bare essentials -- CalculateField one time and export one PDF. I'm now certain the problem is related to the CalculateField_management command. If I comment it out, I get a PDF with the data points visible. If I put it back, the data points disappear from the PDF.

arcpy.CalculateField_management(inFeatures, FieldName, expression, "PYTHON3")

Another point of interest - if I have the attributes table open, it fails to write the PDF.

My theory now is that when I perform CalculateField within the Python Toolbox environment, something gets a lock on the shapefile and fails to release it so the shapefile information gets written to the PDF. Is that plausible?

    def execute(self, params, messages):
                
        # Set path to current AGP project
        p = arcpy.mp.ArcGISProject("CURRENT")
        
        arcpy.CalculateField_management("Anuran_EB_2021_FOSU_OccupancyB", "Occupancy", "!DRCIN!", 'PYTHON3')

        
        ### 1.1 Define PDF export parameters.
        ###
        outputfolder = "C://Users//myname//OneDrive - DOI//Documents//REPORTS//Expanded Briefs//Drafts//Anuran//2021_FOSU//maps//"
        out_pdf = outputfolder+"Anuran_EB_2021_FOSU_333_DRCIN.pdf"
        resolution = 600
        image_quality = "BEST"
        compress_vector_graphics = True
        image_compression = "JPEG"
        embed_fonts = True
        layers_attributes = "LAYERS_ONLY"
        georef_info = False
        jpeg_compression_quality = 100
        clip_to_elements = False
        output_as_image = False
        embed_color_profile = True
        pdf_accessibility = False


        ### 1.2 Export PDF.
        Layout = p.activeView
        Layout.exportToPDF (out_pdf, resolution, image_quality, compress_vector_graphics, image_compression, embed_fonts, layers_attributes, georef_info, jpeg_compression_quality, clip_to_elements, output_as_image, embed_color_profile, pdf_accessibility)
    
        return

UPDATE I've done some additional tinkering and reduced the code to the bare essentials -- CalculateField one time and export one PDF. I'm now certain the problem is related to the CalculateField_management command. If I comment it out, I get a PDF with the data points visible. If I put it back, the data points disappear from the PDF.

arcpy.CalculateField_management(inFeatures, FieldName, expression, "PYTHON3")

Another point of interest - if I have the attributes table open, it fails to write the PDF.

My theory now is that when I perform CalculateField within the Python Toolbox environment, something gets a lock on the shapefile and fails to release it so the shapefile information gets written to the PDF. Is that plausible?

    def execute(self, params, messages):
                
        # Set path to current AGP project
        p = arcpy.mp.ArcGISProject("CURRENT")
        
        arcpy.CalculateField_management("Anuran_EB_2021_FOSU_OccupancyB", "Occupancy", "!DRCIN!", 'PYTHON3')

        
        ### 1.1 Define PDF export parameters.
        ###
        outputfolder = "C://Users//myname//OneDrive - DOI//Documents//REPORTS//Expanded Briefs//Drafts//Anuran//2021_FOSU//maps//"
        out_pdf = outputfolder+"Anuran_EB_2021_FOSU_333_DRCIN.pdf"
        resolution = 600
        image_quality = "BEST"
        compress_vector_graphics = True
        image_compression = "JPEG"
        embed_fonts = True
        layers_attributes = "LAYERS_ONLY"
        georef_info = False
        jpeg_compression_quality = 100
        clip_to_elements = False
        output_as_image = False
        embed_color_profile = True
        pdf_accessibility = False


        ### 1.2 Export PDF.
        Layout = p.activeView
        Layout.exportToPDF (out_pdf, resolution, image_quality, compress_vector_graphics, image_compression, embed_fonts, layers_attributes, georef_info, jpeg_compression_quality, clip_to_elements, output_as_image, embed_color_profile, pdf_accessibility)
    
        return
1
  • Edited to add a photo example. Also, if you know frog taxonomy, yes I am aware the scientific name is messed up and have since corrected it. Oct 4, 2022 at 19:34

1 Answer 1

1

When I have encountered issues like this before, it's usually because I've set some sort of definition query that is preventing the layer from drawing properly, but I don't see a query being set here.

One part of your toolbox code that jumps out at me is that you're calculating a field within the same feature class for each iteration through the species. Is it possible you need to be changing the feature class you're doing the calculation on in each iteration? Otherwise you are just continuously overwriting the same field, which seems odd to me.

I also tend to test my toolbox scripts by hardcoding the input parameters and running the script in a separate IDE, then you can go step-by-step and uncover where bugs occur. I know you've tested the export code as a separate file, but I find it important to also test loops for logic errors.

2
  • I can have up to 60-70 species for which I'm producing maps. What's worked for me is to have a field called "Occupancy" with the symbology set up how I want it, then I import the data for each species into that field one at a time and pop out maps. It's probably not the most efficient way, but it has been working for me for the last year. My next step will be to strip out most of the script and add it back piecemeal, until I get a failure. I have a feeling ArcGIS is doing something differently under the hood when I use the toolbox. I've run it with the loops in Py Window with no probs. Oct 4, 2022 at 18:53
  • I see, that makes sense. I've definitely encountered similar issues before, where the toolbox version simply won't work like the stand-alone script does. In those cases, I have to just leave it as stand-alone even thought it makes it harder for colleagues without Python knowledge to run.
    – David
    Oct 4, 2022 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.