How to set up geoprocessing service to fill out PDF form from within Web AppBuilder

Question

I am trying to get Web AppBuilder (version 2.4) to fill out and return a PDF file as a downloadable link.

I have ArcGIS Desktop 10.4.1 and ArcGIS Server 10.4.1 installed on the same machine.

I have installed pdftk server, a 32-bit command line application for working with PDFs. I have also installed pypdftk inside both C:\Python27\ArcGIS10.4 and C:\Python27\ArcGISx6410.4. The python script works as expected from within ArcMap (32-bit application) and generates a test PDF file that's been filled out based on user input. The outPDF parameter is set as a File data type, type Derived, direction Output.

It lets me publish this as a geoprocessing service to ArcGIS Server. However, when I try to run the geoprocessing service via the geoprocessing widget from within Web AppBuilder, it gives the error: returned non-zero exit status 1 Failed to execute (MakePDF01). Failed to execute (MakePDF01). It gives the same error when running the GP tool/service from within ArcMap.

Is ArcGIS Server, being 64-bit application, capable of running the 32-bit pdftk command-line application? Answer: YES, script runs fine using 64-bit python And if so, can Web AppBuilder (via the geoprocessing widget) generate a PDF file at some specified location on the server?

import arcpy
import os
import sys
import pypdftk

inText1 = arcpy.GetParameterAsText(0)            # String
inText2 = arcpy.GetParameterAsText(1)            # String
inText3 = arcpy.GetParameterAsText(2)            # String
inText4 = arcpy.GetParameterAsText(3)            # String
inText5 = arcpy.GetParameterAsText(4)            # String

field_names = ["Text3", "Text2", "Text1", "Text5", "Text4"]
all_fields = []

all_fields = {
    'Text3': inText1,
    'Text2': inText2,
    'Text1': inText3,
    'Text5': inText4,
    'Text4': inText5
}

outPDFFile = pypdftk.fill_form('C:/Some_Folder/input_pdf.pdf', all_fields, out_file='C:/Some_Folder/output_pdf.pdf')

arcpy.SetParameter(5, outPDFFile)

Full error message in the log: Error executing tool. MakePDF01 Job ID: jc98b008557a14090813562a59bf30cdd : Traceback (most recent call last): File "C:\arcgisserver\directories\arcgissystem\arcgisinput\I24_TDOT\MakePDF01.GPServer\extracted\v101\p02\code_makepdf_01.py", line 33, in outPDFFile = pypdftk.fill_form(g_ESRI_variable_1, all_fields, out_file=g_ESRI_variable_2) File "C:\Python27\ArcGISx6410.4\lib\site-packages\pypdftk.py", line 75, in fill_form run_command(cmd, True) File "C:\Python27\ArcGISx6410.4\lib\site-packages\pypdftk.py", line 42, in run_command p = check_output(command, shell=shell) File "C:\Python27\ArcGISx6410.4\lib\site-packages\pypdftk.py", line 36, in check_output raise subprocess.CalledProcessError(retcode, cmd) CalledProcessError: Command 'pdftk C:\arcgisserver\directories\arcgissystem\arcgisinput\I24_TDOT\MakePDF01.GPServer\extracted\v101\..\cd\p02\input_pdf.pdf fill_form c:\users\arcgis\appdata\local\temp\tmpu_s3h1 output C:\arcgisserver\directories\arcgissystem\arcgisinput\I24_TDOT\MakePDF01.GPServer\extracted\v101\..\cd\p02\output_pdf.pdf flatten' returned non-zero exit status 1 Failed to execute (MakePDF01). Failed to execute (MakePDF01).

UPDATE: Getting the same error message after changing the python code as per Kevin's suggestion:

pdfOutput = os.path.join(arcpy.env.scratchFolder, "output_pdf.pdf")
outPDFFile = pypdftk.fill_form('C:/Some_File/input_pdf.pdf', all_fields, out_file=pdfOutput)

arcpy.SetParameter(5, outPDFFile)

UPDATE 2: When publishing the geoprocessing service, it uploads 'output_pdf.pdf' file to the server. Could this be the issue? The python script should be generating a new output_pdf.pdf each time it is run, therefore it shouldn't need to upload output_pdf.pdf when publishing. It should only need to upload the input_pdf.pdf file.

Also, shouldn't the output PDF show up as type DEFile rather than GPDataFile when setting up the geoprocessing widget?

UPDATE 3: Once the GP Service is published, it looks like ArcGIS Server modifies the python script. Specifically, at the top of the script:

g_ESRI_variable_1 = os.path.join(arcpy.env.packageWorkspace,u'..\\cd\\p02\\input_pdf.pdf')

"When a geoprocessing result is shared as either a geoprocessing package or service, all tools, scripts, datasets, and so on, are consolidated into a single folder. During consolidation, the tool being shared is updated with Package Workspace to ensure all tools, scripts, datasets, and so on, can be located when the shared task is reexecuted. Therefore, during reexecution, Package Workspace is set to the local folder that contains all needed elements to ensure that the task can be completed successfully."

Anyway, it appears that this bait and switch that AGS does is causing the error. Further down, the following line has been modified in the GP service script:

outPDFFile = fill_form(g_ESRI_variable_1, all_fields)

At this point my thinking is that pdftk is not able to handle this g_ESRI_variable_1

Check out the form_fill function inside pypdftk:

https://github.com/revolunet/pypdftk/blob/master/pypdftk.py

Looks like I'll have to modify pypdftk somehow to handle this bait and switch that AGS is doing.

I've spent 3 weeks now trying to get this seemingly simple python script to work with Web AppBuilder.

Answer 1

This is a partial answer; I'll add more after you provide further details.

There is a chance the service doesn't like how you've created your pdf output. You're not using any arcpy functions, but trying to make a file on disk. You should be writing GP Service outputs to scratch as a best practice.

Change the following code from

outPDFFile = pypdftk.fill_form('C:/Some_Folder/input_pdf.pdf', all_fields, out_file='C:/Some_Folder/output_pdf.pdf')

to

pdfOutput = os.path.join(arcpy.env.scratchFolder, "output_pdf.pdf")
outPDFFile = pypdftk.fill_form('C:/Some_Folder/input_pdf.pdf', all_fields, out_file=pdfOutput)

Edit- After some research on pypdftk, it looks like its a full windows application and has some sort of Python interface to it. The program appears to be 32bit only. I think you'll need to do some troubleshooting outside of the "ArcGIS" world here to ensure it can be run as 64bit. Meaning take your code and run it under your 64bit Python: C:\Python27\ArcGISx6410.4 without any arcpy/arcgis inside it. If that works, I'd suspect it should work as a gp service.

Edit2 Thinking a little more about the error you're getting:

Command 'pdftk ......' returned non-zero exit status 1 Failed to execute (MakePDF01). Failed to execute (MakePDF01).

This error to me sounds like python saying "I called the pdftk program, something happened, it told me '1'". It looks like a subprocess type call, where Python is expecting a status code=0 (program quit as expected), and that didnt happen. So why could that happen? Well, ArcGIS Server runs under a user account (generally a local user account to that machine with only certain rights and permissions). Its quite possible that user account does not have adequate permissions to run your PDF output program. (When you call the GP Service, The ArcSOC.exe process calls 64bit Python which is calling the PDF program, this all happens under the account of the arcsoc process) When I suggested you test that it works in 64bit Python (not using the server), you were probably running it under your user account.

What next? Well you could try 1 of 2 things:

• launch python (or a command window to launch python) using the account of the arcgis server. Can you execute the script
• change arcgis server to run under your user account (this is a bit heavy handed, but will answer the question if this is a permission issue)

One last note - you talk about output paths using the WAB. If the output parameter type in your TBX is set to "file" (not string), if the GP Service works successfully it'll return a download link to the WAB. The WAB is then smart enough to make that link clickable for the end user. In short, dont over complicate it by having the script tool return a string. Return a file.