Troubleshooting slow IIS and hanging ArcGIS Server?

Question

Our ArcGIS Server 10.0 SP 4 website has been hanging lately where everything gets delivered except the map. When I try and check the REST Endpoints they it hangs and then fails. When I stop the IIS, that takes 2-3 minutes to stop, which is not normal. Sometimes it takes 2 or 3 stop/starts of IIS to get the server back working again. I've made sure ArcGISWebServices is in Admin group as well as all necessary server groups.

I looked in the system log for the latest crash and it reads:

@ 8:18 AM "A worker process with process id of '13516' serving application pool 'ArcGISApplicationsAppPool' has requested a recycle because the worker process reached its allowed processing time limit.

@8:30 (approx when I began investigating the issue) "A worker process '7616' serving application pool 'ArcGISApplicationsAppPool' failed to stop a listener channel for protocol 'http' in the allotted time. The data field contains the error number."

Any ideas on what I can do to further troubleshoot are appreciated.

Specs. WinServer 2008 IIS 7 AGS 10 SP4

Answer 1

This sounds like a common issue with ArcGIS Server 10 and IIS.

Please follow the steps in the following whitepaper from Esri. These will configure the ArcGIS Web services and REST applications to run under a separate IIS application pool with the identity of the ArcGIS Web services user and disable per request impersonation.

Configure ArcGIS Server REST API for the Microsoft .NET Framework to improve stability and performance

This paper is for Windows Server 2003, but may aid other users that stumble across this Q&A.

On Windows 2003 Server, the Local Security Authority Subsystem Service (lsass.exe) grows in CPU usage and memory utilization under heavy load