8

The issue I face has the following background:

We are geocoding an immense directory of records. It is something in excess of 100 million addresses. I have split the addresses up into as small a geographic reason as I feel is feasible, states. Even so, a single state can have in excess of 5 million records to geocode. The arcpy script I composed will loop through each state's addresses and run the proper geocoding process with the appropriate locator.

The problem:

Sometimes it takes 36 hours, sometimes it takes only 20, but the script will stop with an error I have never seen before (this is from the XML log file):
ERROR 001143: Background server threw an excecption.

  • If the script is run in ArcCatalog or Arcmap's Python window, it will show a red error message Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 001143: Background server threw an excecption.
  • If the script is run in IDLE, it will simply cease processing without an error and restart the shell (with the typical =====RESTART===== heading).

I know this arcpy script works with smaller datasets, as I have been using it for months now. What might be the cause of this error? Do I need to split up my address listings into smaller amounts for them to geocode reliably?

EDIT: I tracked down the Event Viewer Logs per @D.E. Wright and this is what I found listed under the most recent failure

Faulting application name: pythonw.exe, version: 0.0.0.0, time stamp: 0x4ba3e4e2
Faulting module name: Geocoding.dll, version: 10.0.1.2800, time stamp: 0x4cbcbb71
Exception code: 0xc0000005
Fault offset: 0x000be1f3
Faulting process id: 0x%9
Faulting application start time: 0x%10
Faulting application path: %11
Faulting module path: %12
Report Id: %13

Not exactly illuminating...

|improve this question
  • Are you running this with a Server services that you are geocoding against? It sounds like it; and if so you may be seeing Web Services timeout or transaction count that is recycling the services breaking your process. – D.E.Wright Jan 10 '12 at 22:41
  • All the data is local, I'm afraid, which is what confuses me so much. – Nathanus Jan 10 '12 at 22:43
  • Are you using ArcGIS 10? There is a updated process when the software runs, you can see it in your task-manager called ARCSOCM.exe which is essentially a ESRI Server process running locally on your machine to handle the background process. What you may be seeing is this process failing or blowing up on you; you might try checking your event viewer on the machine and seeing if there are any app errors being logged; that can sometimes give information. – D.E.Wright Jan 10 '12 at 22:59
  • Sorry, they are called ArcSOCP.exe and ArcSOMP.exe that are these local processes that run. – D.E.Wright Jan 10 '12 at 23:31
  • 1
    My locators are state-sized. I would probably do the preprocessing with a *nix split command to simply split the text files into 1-2 million record-sized chunks before turning them into tables. Although the extra step of making them tables may be a waste of time since you can geocode text files. – Nathanus Jan 11 '12 at 19:20
1

Here is a white-paper from 09 that has some good topics too about getting your data setup to be more efficient - http://esri.com/library/whitepapers/pdfs/arcgis-server-in-practice.pdf - and here is the link to the download that should be more helpful - http://resources.arcgis.com/gallery/file/geocoding/… - You should find both to help you a lot.

You can also message me direct with more questions since we do a lot a full-state scale processes and are always looking to collaborate.

|improve this answer
  • I think I could use some discussion with someone familiar with ESRI's geocoding "standards." How might I contact you? – Nathanus Jan 25 '12 at 15:46
  • You can find my gmail address under my profile, we can start there. – D.E.Wright Jan 25 '12 at 17:02
0

Perhaps this answer will help you. 100 million addresses shouldn't take much more than a day to process and geocode. Keep in mind that we are not only geocoding the addresses but standardizing them and verifying them as well. If geocoding were the only task, it would be even faster.

|improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.