Add Item and Publish Service with ArcREST or ArcPy

Question

I would like to have an automated process which does the following:

1. Adds a CSV to our portaL/AGOL organization account.
2. Publishes the item as a hosted feature service.
3. Consumes hosted feature back unto EGIS environment.

I have checked out ArcREST as a possible solution, however, one hang up that I can imagine is the recurring ID number that will change as the added CSV is updated on a daily basis via task scheduler.

Is this solution possible with ArcREST?

I will post my code sample when I return to my desk.

I have used a piece of your code Kevin in addition to the addItem method from ArcREST. My item adds, and my token and item ID print, however, when I try to pass the itemID to the function below it does not publish and no errors are returned.

import arcrest
import json
import requests
import urllib
import urllib2
if __name__ == "__main__":
    username = "myusername"
    password = "mypassword"
    portalId = "myportalnumber"
    url = "http://myorg.maps.arcgis.com/"
    thumbnail_url = ''
    file = "myflie.csv"

    securityHandler = arcrest.AGOLTokenSecurityHandler(username,
                                                       password)
    #   Create the administration connection
    #
    admin = arcrest.manageorg.Administration(url, securityHandler)
    #   Connect to the portal
    #
    portal = admin.portals(portalId)
    #   Access the content properties to add the item
    #
    content = admin.content
    #   Provide the item parameters
    #

    itemParams = arcrest.manageorg.ItemParameter()
    itemParams.thumbnailurl = thumbnail_url
    itemParams.title = "CSVTest"
    itemParams.type = "CSV"
    itemParams.tags = "test"
    #   Enter in the username you wish to load the item to
    #
    usercontent = content.usercontent(username=username)
    # print usercontent.addItem(filePath=file,
    #                           itemParameters=itemParams, overwrite=True)
    agolID= usercontent.addItem(itemParameters=itemParams)['id']

    print agolID

    itemId = agolID
    tags = "Demo, Publishing"
    securityHandler = arcrest.AGOLTokenSecurityHandler(username,
                                                       password)






### Generate Token ###
gtUrl = 'https://www.arcgis.com/sharing/rest/generateToken'
gtValues = {'username' : 'myuser',
'password' : 'mypass',
'referer' : 'http://www.arcgis.com',
'f' : 'json' }
gtData = urllib.urlencode(gtValues)
gtRequest = urllib2.Request(gtUrl, gtData)
gtResponse = urllib2.urlopen(gtRequest)
gtJson = json.load(gtResponse)
token = gtJson['token']

print token




def PublishService(Short, agolID, username, token, itemName, publishParams, XField, YField):
    ''' Publishes the input itemID (uploaded CSV) this is using
    a post request from urllib, the JSON input can be generated by
    the analyze function. Item name, X and Y fields are specified'''

    publishURL = 'http://myOrg.maps.arcgis.com/sharing/rest/content/users/myUser/publish'.format(Short, username)
    publishParams['CSVTest'] = itemName
    publishParams['locationType'] = 'coordinates'
    publishParams['Y_COR'] = YField
    publishParams['X_COR'] = XField
    query_dict = {
        'itemID': agolID,
        'filetype': 'csv',
        'f': 'json',
        'token': token,
        'publishParameters':publishParams}

    query_dict['publishParameters'] = json.dumps(query_dict['publishParameters'], sort_keys=False)
    request = requests.post(publishURL, data=query_dict)
    print request.json()`

Answer 1

I just uploaded a script uploads a CSV and publishes as a github-gist. You may want to change the input as right now its setup to run at command line and enter parameters manually when you execute.

The credit goes to a co-worker of mine who shared this with me. I haven't extensively tested it, but it worked for the case I was trying. Note that it does rely on the requests module. You'll need to make sure you've downloaded and installed that.

Also note the script isn't setup to re-publish or update/overwrite. I haven't done any work with re-publishing of a CSV file, but my best guess is you'll need to:

• delete the CSV from the day before (unless you can overwrite it, then this isn't necessary)
• upload the new CSV (that is in the gist)
• Call publish on the new CSV (that is in the gist)
• Set overwrite:true in publish. (that is NOT in the gist). See the doc on publish params here
• You should maintain the original ID because of the overwrite (I haven't tested this to verify)

I should also note, because you asked in your question, there are no true arcpy functions to upload a CSV to Portal. There are functions in ArcGIS Pro to create webmaps or weblayers and the UploadService Defintion tool, but that requires features in a map. Not a stand alone CSV.