1

I have a large 1.1gb CSV table of points in Lat/Long WGS84 for which I'd like to generate a point density raster. I'm taking the following steps but the process is extremely slow. I welcome any advice on how to improve this or better understand where exactly my bottlenecks are.

In ModelBuilder I have the following steps

  1. XY Table to Points - output to in_memory workspace;
  2. Project - reproject in_memory output from LatLong WGS84 to WGS84 Web Mercator (Auxiliary Sphere) - here I'm unable to store the output in_memory (not sure why) so I write it to the gdb;
  3. Point Density - calculate my point density raster from the output of 2.

I'm running on Windows Server 2019 with Intel Xeon Platinum 8175M CPU @2.5GHz (2 processors), 760GB RAM, 64-bit OS which I've spun up an AWS instance with 500GB hard disk.

Currently my ModelBuilder is on step 1 at 1% and it has been there for a while. My CPU usage is at 1% and Memory also at 1%

How can I speed up my process?

  • Try writing your CSV to a file geodatabase instead of in_memory? – Hornbydd Feb 24 at 0:30
  • How long does each step take if run manually using the same tool and parameter values in the Geoprocessing pane? – PolyGeo Feb 24 at 0:34
  • @PolyGeo Not sure what you mean by manually. I built my model and saved it and then hit run from the geoprocessing window. it took 4h15m to run step 1, 1h15m for step 2, and 15min for step 3. – val Feb 24 at 7:29
  • @Hornbydd if I write to the gdb it takes just as long or longer. – val Feb 24 at 7:31
  • 1
    Running manually means not using ModelBuilder while you learn more about how each tool performs by testing them individually. If the first step is your biggest concern then I think you should focus this question on that. If your second step is also of concern then you could start a second question to focus on that. To reduce the time of your testing cycles I think you should switch your testing over to a subset of your data. – PolyGeo Feb 24 at 7:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.