ArcGIS Server will not start on new amazon EC2 windows instance

Question

We have had ArcGIS server running on an EC2 instance without problems for the last 2 years. It is currently at version 10.2. I am now trying to change to a larger instance. I followed Amazons steps to create a new instance

1. Stop existing instance
2. Make a new image from the instance
3. Launch the new image specifying the new larger instance size

I then tried reauthorizing the software which completed successfully (on a side note should this even be necessary since the images are supposed to be identical?) Everything appeared to go smoothly. I can log into the new instance and all data and files are there. The problem is ArcGIS server will not start. I tried manually starting the service from the service console but it stops after a few seconds. It doesn't look like the tomcat web server gets launched. service_error.log shows the following

INFO: getting data at url: http://169.254.169.254/latest/meta-data/local-ipv4 com.esri.arcgis.discovery.admin.AdminException: Could not create directory path '\SITEHOST\config-store'. This is required to maintain the Site configuration. Please check that the location is valid and that the ArcGIS Server account has permissions to the location.

SITEHOST is the name of the server machine in ArcGIS server. Typing \SITEHOST\config-store into a browser shows page not found, but in the working instance it shows a directory with the expected config files. This is why I think the problem is with tomcat starting. Does anybody know what could be causing this?

Answer 1

The answer to my problem was that I had an incorrectly configured host file. ESRI amazon instances have a host entry

SITEHOST    <InternalIPAddress>

When I created a new instance the hosts file still had the SITEHOST entry pointed to the original instances IP address. When I updated the host file so that SITEHOST was now pointing at the new internal IP ArcGIS Server finally came back up.

However this lead to new problems. ArcServer would not start any services because of a security issue with the configstore. Somehow the old config store could not be read by the new instance. I think this had to do with encryption keys but wasn't able to get to the bottom of it. Eventually, with the help of ESRI, I ended up doing a completely new install of ArcGIS Server on the new instance and then importing the map servers from the old config store. This last part isn't officially supported by ESRI and is a little tricky but it worked.

Finally, as an FYI, this whole approach isn't how ESRI recommends upgrading. They recommend upgrading existing instances instead of creating new ones. So officially what I should have done is stopped the existing instance and then restarted it with the new configuration of hardware.

Answer 2

have seen this same issue on both on-premise servers and AWS on arcgis server 10.3.1 to 10.5

This XML file somehow gets deleted by arcgis server

C:\Program Files\ArcGIS\Server\framework\etc\config-store.connection.xml

Note, there is also a config-store.connection.JSON file which is ok. this JSON file has "\SITEHOST" references. do not change this file.

Example config-store.connection.XML contents

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd">
<properties>
<comment>Configuration Store Connection Information</comment>
<entry key="status">Ready</entry>
<entry key="type">FILESYSTEM</entry>
<entry key="connectionString">path to config store here</entry>
<entry key="class">com.esri.arcgis.discovery.admin.store.file.FSConfigStore</entry>
</properties>

Have also seen arcgis webadapter config files reset.

If you can, set both webadapter and the above XML files to readonly, or remove write permissions, after your platform is configured to prevent some buggy code altering the files.