New answers tagged performance
I don't totally understand the output of your analysis, but it seems like that for each feature being evaluated you're doing a lot fiddling - copying datatsets, joining them, adding fields, deleting them. If you don't need to do them every time do them once outside your loop. Some benchmarking might help you figure out where the bottleneck is. You can do ...
I would confirm as @Vince suggests if your input datasets have spatial indices. If your inputs are Shapefiles then these by default do not have spatial indices and need building. Another suggestion that will speed up your code is to replace the cursor you are using with a cursor from the da module as described here.
The long execution time is relatively normal. I had a script which I ran from the Python window inside ArcMap and that took more than two hours. I realized that when I converted the script to a script tool, the execution time went down to 12 minutes. A script tool is like a toolbox. You connect your script to the tool. You will need to add a few lines in ...
You actually forgot to mention, nor included it in the graph, how you measure "icon size": 1) width/height on the X-axis, 2) pixel area of icon in number of pixels I guess it is 1), based on regular icon sizes. That probably explains your graph: the increase in icon area size as measured in number of pixels is quadratic, each doubling of the width, ...
Aside from SQL query optimization and indexes, which you already covered, there's not much. Here are a few things I can think of which might be useful in certain circumstances: If your records don't change I would expect that to result in a minor performance boost, because CartoDB uses Varnish pretty heavily for caching, so keeping the records in your ...
If the linestring are indeed along the same x but vary on y you can create indices on ST_YMAX() and ST_YMIN() functions like so: CREATE INDEX idx_ymin_ymax ON places (St_ymax(geom),St_ymin(geom)); Next just query the table using St_ymax and St_ymin in the WHERE clause to limit the results.
Top 50 recent answers are included