How do I get JSON data into Python script from a ArcGIS geoprocessing service?

Question

I am using Network Analyst to calculate the travel time from point A to point B. I need to do this for thousands of points, so the obvious solution is automating in Python. While I can write a script that calls the relevant ArcPy tools, I'd like to be able to just call our network analyst webservice and parse its results -- I only need the travel time, not the turn-by-turn directions, the route, or any of the other informative results produced through Network Analyst. I assumed that would be faster.

Based on this answer and the Requests module documentation, I thought this would work:

import requests

response = requests.get('http://[url]/arcgis/rest/services/Click/ClickRouting/NAServer/Route',
                        auth=('[user]', '[password]'))

print response.text
print response.json()

BUT, I'm getting a 401 error that claims I don't have permission, even though I can see it just fine through a web browser. Our ArcGIS server guy says that service is not restricted, and doesn't know why there would be an authentication error.

<h2>401 - Unauthorized: Access is denied due to invalid credentials.</h2>
<h3>You do not have permission to view this directory or page using the credentials that you supplied.</h3>

Am I not using requests correctly, or is this a service configuration issue?

(If this doesn't have enough to do with GIS, I apologize; either migrate, or I will ask again on StackOverflow.)

Answer 1

Have a look at my restapi package on GitHub. I have a class for a GpTask. Here's a snippet from my samples page on how to run a GP task (note, not all the tests work right now due to changes to some of the servers, but the GP sample does work):

import restapi

# Test Geoprocessing Service
gp_url = 'http://sampleserver1.arcgisonline.com/ArcGIS/rest/services/Network/ESRI_DriveTime_US/GPServer/CreateDriveTimePolygons'
gp = restapi.GPTask(gp_url)

# get a list of gp parameters (so we know what to pass in as kwargs)
print '\nGP Task "{}" parameters:\n'.format(gp.name)
for p in gp.parameters:
    print '\t', p.name, p.dataType

point = {"geometryType":"esriGeometryPoint",
         "features":[
             {"geometry":{"x":-10603050.16225853,"y":4715351.1473399615,
                          "spatialReference":{"wkid":102100,"latestWkid":3857}}}],
         "sr":{"wkid":102100,"latestWkid":3857}}

# run task, passing in gp parameters as keyword arguments (**kwargs)
gp_res = gp.run(Input_Location=str(point), Drive_Times = '1 2 3', inSR = 102100)

# returns a GPResult() object, can get at the first result by indexing (usually only one result)
# can test if there are results by __nonzero__()
if gp_res:
    result = gp_res.results[0]

    # this returned a GPFeatureRecordSetLayer as an outputParameter, so we can export this to polygons
    print '\nOutput Result: "{}", data type: {}\n'.format(result.paramName, result.dataType)

    # now export the result value to fc (use the value property of the GPResult object from run())
    drive_times = os.path.join(folder, 'drive_times.shp')
    restapi.exportFeatureSet(drive_times, gp_res.value)

The arguments of the "run" method are keyword arguments to match the input parameters of the GpTask you're trying to run. If you're running a GpTask that is secured, you can pass in a username and password like this:

usr = 'username'
pw = 'password'
gp = restapi.GPTask(gp_url, usr, pw) 

This generates a token and uses it for all subsequent requests. Optionally, the 4th argument of any Service object is a token that can be used instead of username and password (must be active token).

P.S. I update this several times a month, so it is good to go back and re-download the current version often. Help can be found here.