Selecting more than 1,000 features from AGOL hosted feature service?

Question

When data from a cloud-hosted feature service is loaded into ArcMap, it queries the first 1,000 features "to enhance performance"... I, however, need to see all the features. How can I add all features to my map? I want to do a selection on the service directly according to this blog post: https://blogs.esri.com/esri/arcgis/2013/10/10/quick-tips-consuming-feature-services-with-geoprocessing/

Max Records

If you’ve already performed some analysis on a hosted feature service from arcgis.com, you may have encountered the following warning:

WARNING 001406: Query operation has exceeded the feature service transfer limit

Only the first 1000 records will be used by the geoprocessing tools. Hosted feature services have their maximum query set to 1000 records. 1000 is also the default maximum with a local ArcGIS for Server. However you have the option to increase this value in your local deployment. To work with a feature service that has more than 1000 records you will have to iterate through the records 1000 (or less) at a time. Remember that geoprocessing tools honor selection sets. You can manually select 1000 records at a time, or you can use a geoprocessing tool like Select Layer by Attribute. Tools like Merge and Append can be used to combine all your features into a single feature class depending on your needs.

However, when I go to use the tool, Select Layer By Attribute, I am unable to use the hosted feature service directly as my Input layer. Is there a specific way I have to do this? Has anyone created a custom tool or script to do this?

Answer 1

Log in to www.arcgis.com with administrator credentials. Click My Content, and navigate to the intended feature service. Open the feature service.

If the error, "There are no services available" is returned when visiting the ArcGIS REST Administrator Directory, the hosted feature service is shared with 'Everyone' (public).

In the item details page of the feature service, click the hyperlink under 'Layers'. The ArcGIS REST Services Directory page opens. Modify the URL to the one displayed below, appending 'admin' to the end of the URL and deleting the rest of the URL to the service:

http://services.arcgis.com/xxxx/ArcGIS/admin

Click Enter to access the new page. Click Services and find the intended service. Click the service name. The URL becomes:

http://services.arcgis.com/xxxx/ArcGIS/admin/services/states.FeatureServer

Scroll to the bottom of the page, and click Update Definition. Change the value of the lastEditDate key to null.

Search for the maxRecordCount parameter (Press Ctrl + F and search for 'maxRecordCount'). Change the values to the number of desired features records to display. Click Update Service Definition.

Go back to the 'My Content' page and open the feature service in a new web map. All features beyond 1000 are now included and displayed as desired.

Answer 2

If you are looking to extract all the features from a ArcGIS Server MapService for use in ArcGIS Desktop, you can use chupaESRI. The script will extract all features from a MapService end point and store the features in a PostGIS database. From there, you can access the downloaded data like any other geodatabase feature class.

The tool works around the 1000 feature limit by making multiple requests and appending the features. It also pulls from a MapService, so it will work even if the FeatureService is disabled.

Full disclosure, this is a project of mine. I welcome feedback and suggestions for improvements.