Reading local file from Geoprocessing Service on ArcGIS Server

Question

I have a Python toolbox which I would like to publish to our ArcGIS Enterprise 10.8.1 environment. I use ArcGIS Pro 2.7.3 for publishing. The Python toolbox reads from a configuration file on the file system.

After publishing and running the geoprocessing service on the server, the configuration are based on the file on my local machine rather than the configuration file on the server (in the same directory) which is not desired.

import json
# ...

DATASOURCE_CONFIG_FILE = r"f:\geoportal\data.json"

class Tool(object):
    # ...

    @staticmethod
    def read_datasources():
        """ Read datasource config file. """
        with open(DATASOURCE_CONFIG_FILE, "r") as config_file:
            return json.load(config_file)

I assume this is some sort of a security mechanism (sandbox?). Could someone explain me what ArcGIS Pro and/or ArcGIS Enterprise is actually doing/changing when publishing a tool and why?

Also, is there another workaround other than "obfuscating" DATASOURCE_CONFIG_FILE = r"f:\geoportal\data.json" to conceal it from the publishing parser?

---

Doing some more research I found this:

Whenever a quoted string is found in the script, the test for data existence proceeds as follows:
[...]
These tests proceed in sequential order. If the test passes, and the data exists, it will be consolidated, with one exception: if you are sharing a service, the server's data store is examined to determine if the data resides in the data store. If it resides in the data store, then it is not consolidated.

Authoring geoprocessing tasks with Python scripts

In my case, I have a configuration file on my local machine and on the server machine located in the same directory (but different file content). So, in this case it should not be consolidated, right?

---

Solution: See KHibma's answer and my comment. The issue was that I used localhost instead of MY_COMPUTER_NAME for the Publisher Folder Hostname when registering a folder in Data Stores. localhost does not work despite of being mentioned here.

Field	Value
Publisher Folder Path	f:\geoportal
Publisher Folder Hostname	MY_COMPUTER_NAME
Server Folder Path	Same as publisher folder path checked

Answer 1

Based on how you've presented your file in code: DATASOURCE_CONFIG_FILE = r"f:\geoportal\data.json" I would assume that the publishing process would find and identify this as data. Once identified, this file will be subject to the publishing rules. Everything will depend on if you have your file directories setup in the data store. That is, if you've told your Enterprise deployment about f:\geoportal. Assuming you have an entry in your data store, publishing will NOT copy contents of this folder. The publishing process will update paths if needed. If you HAVE NOT created a data store entry, then your file, data.json will be consolidated, copied up to the server, deployed into the working directory of the service (a directory deep within /arcgisserver/arcgisinput/*), and your script powering the service will be updated to point to the file location.

Since you're saying its using the same file as on your desktop, I would speculate that you have NOT created a data store entry for this. As such, it's copying the file and will for now and until the end of time use the version of the file it's copied. Thus, you'll want to create a data store entry so it doesn't copy. Something like:

Local: f:\geoportal

Server: f:\geoportal (looks the same because you've indicated your server mirrors your local directory structure.)

The publishing process will "update" your script data path it publishes to f:\geoportal. That may seem whacky, but it's updating it to the location on the server, which again, just happens to be mirrored in your case.