2

I would like to have an automated process which does the following:

  1. Adds a CSV to our portaL/AGOL organization account.
  2. Publishes the item as a hosted feature service.
  3. Consumes hosted feature back unto EGIS environment.

I have checked out ArcREST as a possible solution, however, one hang up that I can imagine is the recurring ID number that will change as the added CSV is updated on a daily basis via task scheduler.

Is this solution possible with ArcREST?

I will post my code sample when I return to my desk.

I have used a piece of your code Kevin in addition to the addItem method from ArcREST. My item adds, and my token and item ID print, however, when I try to pass the itemID to the function below it does not publish and no errors are returned.

import arcrest
import json
import requests
import urllib
import urllib2
if __name__ == "__main__":
    username = "myusername"
    password = "mypassword"
    portalId = "myportalnumber"
    url = "http://myorg.maps.arcgis.com/"
    thumbnail_url = ''
    file = "myflie.csv"

    securityHandler = arcrest.AGOLTokenSecurityHandler(username,
                                                       password)
    #   Create the administration connection
    #
    admin = arcrest.manageorg.Administration(url, securityHandler)
    #   Connect to the portal
    #
    portal = admin.portals(portalId)
    #   Access the content properties to add the item
    #
    content = admin.content
    #   Provide the item parameters
    #

    itemParams = arcrest.manageorg.ItemParameter()
    itemParams.thumbnailurl = thumbnail_url
    itemParams.title = "CSVTest"
    itemParams.type = "CSV"
    itemParams.tags = "test"
    #   Enter in the username you wish to load the item to
    #
    usercontent = content.usercontent(username=username)
    # print usercontent.addItem(filePath=file,
    #                           itemParameters=itemParams, overwrite=True)
    agolID= usercontent.addItem(itemParameters=itemParams)['id']

    print agolID

    itemId = agolID
    tags = "Demo, Publishing"
    securityHandler = arcrest.AGOLTokenSecurityHandler(username,
                                                       password)






### Generate Token ###
gtUrl = 'https://www.arcgis.com/sharing/rest/generateToken'
gtValues = {'username' : 'myuser',
'password' : 'mypass',
'referer' : 'http://www.arcgis.com',
'f' : 'json' }
gtData = urllib.urlencode(gtValues)
gtRequest = urllib2.Request(gtUrl, gtData)
gtResponse = urllib2.urlopen(gtRequest)
gtJson = json.load(gtResponse)
token = gtJson['token']

print token




def PublishService(Short, agolID, username, token, itemName, publishParams, XField, YField):
    ''' Publishes the input itemID (uploaded CSV) this is using
    a post request from urllib, the JSON input can be generated by
    the analyze function. Item name, X and Y fields are specified'''

    publishURL = 'http://myOrg.maps.arcgis.com/sharing/rest/content/users/myUser/publish'.format(Short, username)
    publishParams['CSVTest'] = itemName
    publishParams['locationType'] = 'coordinates'
    publishParams['Y_COR'] = YField
    publishParams['X_COR'] = XField
    query_dict = {
        'itemID': agolID,
        'filetype': 'csv',
        'f': 'json',
        'token': token,
        'publishParameters':publishParams}

    query_dict['publishParameters'] = json.dumps(query_dict['publishParameters'], sort_keys=False)
    request = requests.post(publishURL, data=query_dict)
    print request.json()`
7

I just uploaded a script uploads a CSV and publishes as a github-gist. You may want to change the input as right now its setup to run at command line and enter parameters manually when you execute.

The credit goes to a co-worker of mine who shared this with me. I haven't extensively tested it, but it worked for the case I was trying. Note that it does rely on the requests module. You'll need to make sure you've downloaded and installed that.

Also note the script isn't setup to re-publish or update/overwrite. I haven't done any work with re-publishing of a CSV file, but my best guess is you'll need to:

  • delete the CSV from the day before (unless you can overwrite it, then this isn't necessary)
  • upload the new CSV (that is in the gist)
  • Call publish on the new CSV (that is in the gist)
  • Set overwrite:true in publish. (that is NOT in the gist). See the doc on publish params here
  • You should maintain the original ID because of the overwrite (I haven't tested this to verify)

I should also note, because you asked in your question, there are no true arcpy functions to upload a CSV to Portal. There are functions in ArcGIS Pro to create webmaps or weblayers and the UploadService Defintion tool, but that requires features in a map. Not a stand alone CSV.

  • @KHimba please see updates – Geoffrey West May 5 '15 at 13:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.