54

That's strange, as if people suddenly discovered the power of Python (without ArcPy which is just one Python module among others), see for example the question Visualize shapefile in Python: geospatial processing in Python has a very long history, much older than Arcpy (or arcgisscripting) -> no "mimic" the capabilities of ArcPy here, as Paul says, most ...


30

You will find a number of other similar questions on this site that ask the same basic question and have very good references. The most similar (and detailed) is: What are the Python tools/modules/add-ins crucial in GIS? Others include: Python Script examples for geoprocessing shapefiles without using arcpy Pure Python Library for Geometry Operations ...


22

We need to bear in mind that these data are samples of discrete lithologic domains. Often, the boundary between two such domains cannot be identified in the field and so it's not valid to expect that many of the sample locations will lie precisely along boundaries. A correct solution will be a partition of the study area and each polygon within that ...


20

A good starting point would be the Geospatial Data Abstraction Library. It is actually made up oftwo libraries -- GDAL for manipulating geospatial raster data and OGR for manipulating geospatial vector data but people usually just call it GDAL. There's a geoprocessing with Python using open source GIS course at the Utah State University. You might want to ...


20

In a lot of my academic research I work with LiDAR data doing surface analysis for geomorphology. I quickly found that performing a lot of operations using arcpy was very slow, especially on large datasets. As a result I began using: pyshp to manipulate shapefiles and update attribute tables numpy to manage ASCII rasters and perform kernel-based analysis ...


19

I highly recommend the USU site Geoprocessing with Python using Open Source GIS to get you started. They primarily use the GDAL/OGR library throughout the exercises. Installing GDAL/OGR can be a bit of a challenge, so this blog entry may be helpful for you: Installing GDAL (and OGR) for Python on Windows. Also check out Alternatives to using Arcpy on GIS....


19

I ran a test to determine how the speed and quality differs between the two methods, here are the results: Input data 4-band NAIP DOQQ image in .img format (349.34MB) A feature class used as the mask/clipper Performance Three trials were performed and benchmarked. The Clip (Data Management) method is significantly faster than the Extract by Mask (...


17

It's not a very well-known feature, but you need the Feature Type Connections window. You access it like this (View > Windows > Feature Type Connections): In there select all the source feature types, select the transformer point to connect to, then click Connect:


16

For people using ESRI I think GRASS would be a very similar environment with a GUI python environment and organized in separate 'toolkits' for different tasks (raster, vector, solar toolkits etc.). The scripting has other options besides Python but that is how I use it. Definitely check out this great link which is up-to-date (I believe): http://grass.osgeo....


16

I think the answers given so far cover basically all package out there worth mentioning (espically GDAL, OGR, pyshp, NumPy) But there is also the GIS and Python Software Laboratory, that hosts a couple of interesting modules. They are: Fiona: OGR's neater API Rtree: spatial index for Python GIS Shapely: Python package for manipulation and analysis of ...


15

Yes, I believe it is possible to use both in Python scripts. XTools has some samples in your install directory (mine is an older version and is installed at C:\Program Files (x86)\DataEast\XToolsPro 7.1\Scripts). You need to import the Toolbox. Here is an example (assuming you are on ArcGIS 10 and using arcpy, otherwise use the geoprocessor): arcpy....


15

While I'm a big user of both shapely and fiona, I wouldn't go this approach. This is a task of writing an effective SQL statement. Using ogr2ogr with an SQLITE dialect, you can process this from a command line. Change directory to one before the shapefiles, so that all of the shapefiles are in one directory called data. OGR treats directories of shapefiles ...


14

With ArcGIS 10.1 it is possible to disable add output to the application display with arcpy.env.addOutputsToMap = False http://resources.arcgis.com/en/help/main/10.1/index.html#//018z0000004s000000


14

The first thing I would do is monitor your system's resource utilization using something like Resource Monitor in Windows 7 or perfmon in Vista/XP to get a feel for whether you are CPU-, memory- or IO-bound. If you are memory or IO-bound there is likely very little that you can do but upgrade hardware, reduce the problem size, or change the approach ...


14

Your problem is likely because of what you have included in the group by field. ST_UNION is an aggregate function, meaning that it is dissolving based on what you specify as the GROUP parameter. Here is what you entered: SELECT c.fid, ST_Union(c.boundaryshape) FROM c Group by c.fid,c.boundaryshape; According to this, you are grouping by your fid, which ...


14

ESRI has a blog post - aptly titled Dicing Godzillas - about processing feature classes with large amounts of vertices. It discusses the Dice tool, which splits up a shapefile's features into smaller features with less vertices. You won't find much of a difference (if any) running in ArcCatalog vs. ArcMap. However, you might find some improvement by ...


14

laspy is another good LAS read/write software. It supports working with the data directly in numpy arrays and a number of other nice Pythonic features. It isn't processing software per se, however. PDAL has the ability to use Python as an in-pipeline filtering language, but this isn't a processing engine either. There isn't too much in the Python quiver ...


13

There is an excellent QGIS plugin called Profile Tool that creates trail profiles. In the first screenshot, I overlaid an OpenLayers Google satellite image over a DEM and hand digitized my path using the profile tool. The second screenshot shows the profile results. You can download DEM's across the USA from the National Map, Earth Explorer or the NRCS ...


13

Note that as of QGIS 2.8, there's a new tool in Processing Toolbox called "Split lines with lines" that does exactly this task and works beautifully.


13

I am not so sure that this is a CPU-bound task. I'd think it would be an I/O-bound operation, so I'd be looking to use the fastest disk to which I had access. If E: is a network drive, then eliminating that would be the first step. If it isn't a high performance disk (<7ms seek), then that would be second. You may achieve some benefit from copying the ...


13

Let me guess: Your cpu has 4 cores, so 25% cpu usage, is 100% usage of one core, and 3 idle cores. So only solution is to make the code multi threaded, but that is no simple task.


12

GDAL has a wonderful file format called VRT, which is an XML wrapper around one or more raster files. One feature of VRTs is their ability to encode square convolution kernels for any given band. It does involve playing around with XML in a text editor (or programatically), but if you're already used to the GDAL tools, it shouldn't be too hard. To ...


12

I thought there must be a way to do this, so I created my what I believe to be a pretty good solution. I have posted it on the ArcGIS Resources site and in the Community->Technical->Analysis & Geoprocessing->Analysis->Gallery. The tool is called Split Polygons With Lines and requires an ArcInfo license because of some of the tools used within the model....


12

You can use the GDAL Python bindings. Examples on how to use it can you find here. For example you create points with lat/lon like this from osgeo import ogr # first import the library point1 = ogr.Geometry(ogr.wkbPoint) point1.AddPoint(13.381348,52.536273) # Berlin point2 = ogr.Geometry(ogr.wkbPoint) point2.AddPoint(11.557617,48....


11

Two options come to mind. If you want a specific LINESTRING then you can use ST_NumGeometries() and ST_GeometryN(). Alternatively, if you want all the sub-geometries, ST_Dump() is the way to go. After actually reading the question, you will need to do something similar to this post from the postgis-users list: SELECT ST_AsText( ST_MakeLine(sp,ep) ) FROM --...


11

you can check out k-means clustering algorithm here. In data mining, k-means clustering is a method of cluster analysis which aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean. This results into a partitioning of the data space into Voronoi cells. kmeans-postgresql implementation ...


11

With the Compare Feature Tool, you should just choose your Segment_ID field as the sort field in the dialog. The [sort] field or fields [are] used to sort records in the Input Base Table and the Input Test Table. The records are sorted in ascending order. Sorting by a common field in both the Input Base Features and the Input Test Features ensures ...


11

Use the Delete Identical tool. There's also a Find Identical tool to check which features have duplicates you can use first, if you want.


11

QGIS uses the Douglas-Peucker algorithm (slightly modified to handle closed loop like polygons, I think) and the unit of the tolerance parameter is the same as the unit of the reference system. Points are removed if the distance with the tentative simplified line is smaller than the tolerance.


Only top voted, non community-wiki answers of a minimum length are eligible