Hot answers tagged arcpy
19
You will find a number of other similar questions on this site that ask the same basic question and have very good references. The most similar (and detailed) is:
What are the Python tools/modules/add-ins crucial in GIS?
Others include:
Pure Python Library for Geometry Operations
What tools in Python are available for doing great circle distance + ...
17
General python optimization techniques can save you substantial amounts of time.
One really good technique for getting a lowdown of where the hold ups are in your script is using the built-in cProfile module:
from cProfile import run
run("code") # replace code with your code or function
Testing using a small data sample will allow you to pinpoint which ...
16
A good starting point would be the Geospatial Data Abstraction Library. It is actually made up oftwo libraries -- GDAL for manipulating geospatial raster data and OGR for manipulating geospatial vector data but people usually just call it GDAL.
There's a geoprocessing with Python using open source GIS course at the Utah State University. You might want to ...
14
I am using an example with 1 million randomly generated points inside of a filegeodatabase. Attached here.
Here is some code to get us started:
import time
import arcpy
arcpy.env.workspace = "C:\CountTest.gdb"
time.sleep(5) # Let the cpu/ram calm before proceeding!
"""Method 1"""
StartTime = time.clock()
with arcpy.da.SearchCursor("RandomPoints", ...
14
GDAL is the tool to use. In fact that entire call is one line for gdal_rasterize:
gdal_rasterize -l mask -i -burn -9999 mask.shp elevation.tif
if you knew the no data value of the dem
For some python control:
lyr = 'mask'
shp = 'mask.shp'
dem = 'elevation.tif'
ndv = -9999
p = os.Popen('gdal_rasterize -l %s -i -burn %d %s %s' % (lyr,ndv,shp,dem)
where ...
14
If you need to create a second cursor for parcels.shp, do so outside of the loop for your first cursor. As it stands, your script is creating a new cursor object for each row in malls.shp which is what's costing you all that processing time.
...
rows = arcpy.UpdateCursor('malls.shp',"","",'ParcelID')
polyrows = arcpy.SearchCursor('parcels.shp')
for row in ...
12
The first thing I would do is monitor your system's resource utilization using something like Resource Monitor in Windows 7 or perfmon in Vista/XP to get a feel for whether you are CPU-, memory- or IO-bound.
If you are memory or IO-bound there is likely very little that you can do but upgrade hardware, reduce the problem size, or change the approach ...
12
The Polyline class has a new method called "positionAlongLine" in ArcGIS 10.1. This will return a PointGeometry object with exactly one point at a specified distance from the starting end of the line, or a fraction of the distance between the start and end. To find the midpoint, you would just need to do positionAlongLine(0.5,True). To find the midpoints for ...
12
Try Googling for "python on error resume next" or similar. This returns a number of hits including this one from StackOverflow:
If you know which statements might fail, and how they might fail, then
you can use exception handling to specifically clean up the problems
which might occur with a particular block of statements before moving
on to the ...
12
In a lot of my academic research I work with LiDAR data doing surface analysis for geomorphology. I quickly found that performing a lot of operations using arcpy was very slow, especially on large datasets. As a result I began using:
pyshp to manipulate shapefiles and update attribute tables
numpy to manage ASCII rasters and perform kernel-based analysis ...
12
You need to loop through your inputs. Multivalue is semicolon delimited. Split on that and loop through them. (AddMessages to show how the fcs are presented)
import arcpy
ins = arcpy.GetParameterAsText(0)
arcpy.AddMessage(ins)
for fc in ins.split(';'):
arcpy.AddMessage(fc)
arcpy.Clip_analysis(fc, clipfeats, out)
Though I'm not entirely sure of ...
12
You shouldn't have to delete the shapefile. Just add the following lines to your script:
import env
arcpy.env.overwriteOutput = True
That will allow arcpy.CreateFeatureclass_management to overwrite the existing shapefile with all of its associated files (dbf, shx, prj etc.).
12
I think the problem is likely your two lines where you are going over the fields and appending each field individually to your subdict dictionary.
for field in valid_fields:
subdict[field] = row[cursor_fields.index(field)]
Your row object is already a tuple in the same order as your fields, take advantage of that and use the zip function.
def ...
11
For people using ESRI I think GRASS would be a very similar environment with a GUI python environment and organized in separate 'toolkits' for different tasks (raster, vector, solar toolkits etc.). The scripting has other options besides Python but that is how I use it.
Definitely check out this great link which is up-to-date (I believe):
...
11
you can use python for getting EXIF info:
from PIL import Image
from PIL.ExifTags import TAGS
from pprint import pprint
def getexif(im):
res = {}
try:
img = Image.open(im)
info = img._getexif()
for tag, val in info.items():
dec = TAGS.get(tag, tag)
res[dec] = val
except IOError:
print im
...
10
Which part of the script is actually taking up most of the time? There are about 5 other steps going on before you actually start deleting stuff.
You might want to break your script down into bite-sized tests. For example, instead of creating a temporary connection file, listing a bunch of datasets, listing their contents, counting their records, and then ...
10
Make sure you are writing to internal drive on the computer. Reaching across the network when it is not necessary can really slow the processing. It can even be faster to copy the data as the first step in the process to keep the subsequent read-writes as quick as possible
Running the script completely outside of ArcMap can be much faster. If a Map isn't ...
10
You are on the right track with your script. It looks like your problem lies in how you are comparing the Layer object, to the Name of the Layer in the Table of Contents.
When you use the ListLayers function, what is returned is a Layer object. You cannot then compare this to a text string to see if they are equal, you need to access the Name of the Layer ...
10
I think the answers given so far cover basically all package out there worth mentioning (espically GDAL, OGR, pyshp, NumPy)
But there is also the GIS and Python Software Laboratory, that hosts a couple of interesting modules. They are:
Fiona: OGR's neater API
Rtree: spatial index for Python GIS
Shapely: Python package for manipulation and analysis of ...
10
You'll notice that TRU_W_DatableFeatures is an optional input. When ArcGIS calls a Python script with optional arguments it will pass in a # in place of an optional argument that has not been filled out. This is because Python arguments are positional.
Otherwise if you were calling this script from Python instead of ArcGIS you (may) not set ...
10
What you need is an UpdateCursor.
The UpdateCursor function creates a cursor that lets you update or
delete rows on the specified feature class, shapefile, or table. The
cursor places a lock on the data that will remain until either the
script completes or the update cursor object is deleted.
Your code would look something like the following:
...
9
If you are going to ArcMap 10.1 you could create a python add-in. The add-in gives you access to an "on open" function that will run code when you open the mxd.
The help here explains how to create one and has a sample that adds a base layer to the mxd when opening.
9
A workaround for not having the TableToNumPyArray command would be this:
First, iterate through all features in your feature class, and store the price data in a Python dictionary.
Second, iterate again through all features in your feature class, and just lookup your price data from the dictionary.
Instead of the processing time for n features being ...
9
I've been using "in_memory" quite a bit recently. It can be very useful, as it has the potential to dramatically increase processing speeds for certain tasks, however if you are working with very large datasets, it might cause your program to crash.
You can use "in_memory" to define process outputs... often, if I am performing a task on a feature class, ...
9
@sgrieve's answer works for ArcMap 10.1 users with the data access (da) module. You can do this in earlier versions of Arc by using row.getValue. I'll modify sgrieve's example:
import arcpy
fc = 'C:\\shp\\islands.shp'
fields = ['FID','Id'] #insert whatever variable you need into this list
cursor = arcpy.SearchCursor(fc)
for row in cursor:
print ...
9
This works for me, using the arcpy.da.Walk function at ArcGIS 10.1 SP1:
import arcpy, csv, os
workspace = r"c:\GISData"
output = r"C:\temp\test.csv"
with open(output, 'wb') as csvfile:
csvwriter = csv.writer(csvfile)
for dirpath, dirnames, filenames in arcpy.da.Walk(workspace):
for filename in filenames:
desc = ...
9
Use the following method on the Result object and you'll be able to cast as int:
.getOutput(0) will return the value at the first index position of a tool.
int(arcpy.GetCount_management(Path_Pts).getOutput(0))
8
This works for me:
arcpy.Clip_management("C12.TIF","481919 5456830 482895 5456851","in_memory/raster2","#","256","NONE")
When I first tried this, I just wrote right into the python window, I got an error. So I ran the GUI tool, and copy/pasted the 'python snippet' from that, it worked. I also tried just 'in_memory', this writes a raster named 'in_memory' ...
8
Here is an example of a multicore arcpy script. The process is very CPU-intensive so it scales very well: Port "Producing Building Shadows" Avenue code to ArcGIS 10
Some more general info in this answer: Can concurrent processes be run in a single model?
8
In my experience the biggest problem is managing stability. If you do six weeks of processing in a single night you will also have six weeks of inexplicable errors and bugs.
An alternative approach is to develop standalone scripts that can run independently and fail without causing problems:
Split the data into chunks that a single core can process in ...
Only top voted, non community-wiki answers of a minimum length are eligible
