Questions tagged [pandas]
pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.
266
questions
79
votes
3
answers
119k
views
Converting Pandas DataFrame to GeoDataFrame
This seems like a simple enough question, but I can't figure out how to convert a Pandas DataFrame to a GeoDataFrame for a spatial join?
Here is an example of what my data looks like using df.head():
...
- 1,549
70
votes
9
answers
81k
views
Finding nearest point in other GeoDataFrame using GeoPandas
I've got two geodataframes:
import geopandas as gpd
from shapely.geometry import Point
gpd1 = gpd.GeoDataFrame([['John',1,Point(1,1)],['Smith',1,Point(2,2)],['Soap',1,Point(0,2)]],columns=['Name','ID'...
- 1,623
20
votes
7
answers
21k
views
Reading raw data into geopandas
Is it possible to read raw data into a geopandas GeoDataFrame, a la a pandas DataFrame?
For example, the following works:
import io
import pandas as pd
import requests
data = requests.get("https:/...
- 1,185
18
votes
1
answer
34k
views
Change marker size in plot with GeoPandas
I have a geodataframe with points and some associated data. I want to plot it on a map using geopandas and have the size of the points correspond to one of the columns in the geodataframe.
So far I ...
15
votes
1
answer
16k
views
Shapely coordinate sequence to GeoDataFrame
Using the shapely bounding box function box, and a GeoDataFrameof points, I have extracted a bounding box and created a buffer around it (below gdf is a GeoDataFrameof points):
bbox=shapely.geometry....
- 2,102
15
votes
1
answer
15k
views
GeoPandas to_file() saves GeoDataFrame without coordinate system
I have
GeoPandas 0.2.1 py27_0
fiona 1.7.0 np110py27_1
Python 2.7.10
installed in Anaconda 2-4.1.1-Windows-x86 environment. I am able to construct GeoDataFrame by reading input datasets and ...
- 30k
14
votes
3
answers
42k
views
Making shapefile from Pandas dataframe?
I´d like to construct a shapefile from a Pandas Data Frame using the lon & lat rows.
I have got a csv file and I process it with pandas to make a data frame which is easier to handle
Is it ...
- 757
13
votes
1
answer
23k
views
TypeError: Input geometry column must contain valid geometry objects
I'm working in a urban biodiversity project and I have a csv file for a birds inventory with a column index called 'geom' composed by multypolygons as this:
name geom ...
- 636
13
votes
1
answer
2k
views
Counting features into intersections of Shapely Polygons
I have a geopandas GeoDataFrame containing hundreds of shapely Polygon and MultiPolygon geometries. The polygons overlap in lots of places. I'd like to make a new geometry that contains a count of how ...
- 231
12
votes
2
answers
12k
views
Quick way to get WKT from a GeoDataFrame's geometries in GeoPandas?
Is there a quick way to produce a column/series containing the WKTs of all the geometries in a GeoPandas GeoDataFrame?
I got around this using the apply function (see example below), but I just ...
- 2,729
10
votes
1
answer
17k
views
Merging a geodataframe and pandas dataframe based on a column
I have a simple geopandas dataframe that includes geometry and a column called 'MUKEY':
merged_spatial_df.head()
I want to merge it to a tabular (.csv) pandas dataframe (which also has a column ...
- 1,933
10
votes
3
answers
16k
views
Geopandas: Write layer back into GeoDataBase
I use geopandas (which uses fiona) to read a GeoDataBase layer (using driver="OpenFileGDB") into a geodataframe for easy manipulation.
read_shp = gpd.read_file(GDB_file, layer=layer_choice)
That ...
- 209
9
votes
3
answers
8k
views
Create pandas DataFrame from raster image - one row per pixel with bands as columns
I have a raster image with 3 bands. I would like to convert this image to a csv file where each row will be one pixel and each column will be one band, so that I can easily see the three values each ...
- 2,207
8
votes
4
answers
24k
views
Adding elevation, name and description in GeoJSON when converting DataFrame with Python
I have this working code, data2kml() function works ok, but data2geojson() is incomplete. How can I add the elevation, name and the description in the GeoJSON? I can't understand how FeatureCollection ...
- 293
8
votes
2
answers
1k
views
"Copy" Image coordinates to another image that is nd.array
I have image with three bands. I have script predicts the first band pixels values by using the two other bands and in the end generates an image.The generated image is at the beginning pandas table, ...
- 2,207
8
votes
2
answers
14k
views
Building GeoDataFrame row by row
I am trying to build GeoDataFrame row by row, from an empty one. The equivalent using only pandas would be something like this :
df = pandas.DataFrame(columns=['a','b','c','d'])
df.loc['y'] = ...
- 363
8
votes
1
answer
2k
views
Creating QgsVectorLayer from Pandas dataframe
I am writing a python script for QGIS.
I have a pandas dataframe that I would like to use in order to create a QgsVectorLayer.
At the moment I am saving the data frame into a .csv and then I load it:...
- 223
7
votes
1
answer
1k
views
Creating polyline from list of QgsPoints using PyQGIS
So I have a CSV of points that I've cleaned up using pandas, and appended to a list with each item being Type: QgsPoint, with its correct X and Y coordinate. I'm trying to find the correct operation ...
- 151
7
votes
1
answer
1k
views
Plotting a CSV with coordinates one row at a time via GeoPandas
I have a CSV of all earthquakes in my country for the last 10 years arranged by date and time. My goal is to plot each line chronologically one by one, so that I can compile these maps and produce a "...
- 1,217
7
votes
1
answer
2k
views
GeoPandas GeoDataFrame plot statistics - how?
The plot method of a GeoPandas GeoDataFrame overrides the DataFrame parent method to plot the geometries of the associated GeoSeries. This is very convenient, but I'd also like to use the easy ways ...
- 1,246
7
votes
0
answers
341
views
QGIS plugin external libraries in plugin’s directory
I’m developing my first QGIS plugin and I have to use two external libraries not included in the default OSGeoW64 installation: Couchdb and Pandas.
I would like to include those libraries in my ...
7
votes
0
answers
1k
views
Import Pandas or any Python module into QGIS internal Python
(Also asked in stack overflow -
https://stackoverflow.com/questions/43244062/import-pandas-or-any-python-module-into-qgis-internal-python )
I am developing a plugin for the current LTR version of ...
- 796
6
votes
2
answers
7k
views
Finding distance of consecutive points in a GeoPandas data frame
I am trying to find the distance between every consecutive points in a GeoPandas dataframe. Let's say we have the following data frame:
import pandas as pd
import geopandas
data=pd.DataFrame()
data['...
- 136
6
votes
2
answers
3k
views
Joining two Pandas DataFrames based on lat, long as fields
I have two Pandas DataFrames containing "lat" and "long" coordinates. I'd like to do a spatial join and merge columns from one DataFrame into another.
import pandas as pd
df1 = pd....
- 481
6
votes
3
answers
6k
views
Read a GDB table into a pandas dataframe in ArcPy
I have a set of tables that I have exported to a file geodatabase. I would like to read it into python as a pandas dataframe. When I try to read the file directly using pd.read_csv
import pandas as pd
...
- 447
6
votes
1
answer
7k
views
Geopandas TypeError when saving a GeoDataFrame after creating CategoricalDtype
I'm trying to prepare a column classification of a GeoDataFrame before exporting to QGIS. So, I use pandas.cut. However, when I want to save it I get a
TypeError: Cannot interpret 'CategoricalDtype(...
- 175
5
votes
1
answer
4k
views
Installing pandas with Esri's python?
Trying to get pandas to import, but I'm getting what looks like an error caused by a failed dependency of the version of my Esri-provided numpy's compile, based on this stackoverflow posting. Here's ...
- 1,290
5
votes
1
answer
16k
views
Getting elevation for multiple lat long coordinates in Python
I have seen the answer to Elevation info for multiple lat long coordinates and I would like to be able to do this with a python script instead. I cannot use Googles API because I have 8000 points. I ...
- 223
5
votes
1
answer
898
views
Converting Pandas column with featurecollection to GeoJSON
I downloaded a CSV that contains a column which has a GeoJSON format and imported it as Pandas DataFrame. How can I convert this to a GeoJSON .geojson? I have about 10,000 rows, each with information ...
- 215
5
votes
1
answer
14k
views
Groupby Ploting - give each plot title/name
I have used this code to group by the continents and then plotted using geopandas. I would like to set the plot title for each continent. I would have liked also to draw the continents side by side.
...
- 333
5
votes
1
answer
465
views
Read pixel values of a raster image for given coordinates stored in a CSV file
I was trying to find the pixel values of several pixels whose coordinates are known. I did that in this way:
import rasterio
lons = [66.979, 69.2979, 67.2979 ]
lats = [36.5854, 40.9854, 37.9854]
...
- 196
5
votes
1
answer
326
views
import anaconda-installed modules from arcgis's python
How can I import anaconda-installed modules from arcgis's python shell? I have tried the method described in https://sites.google.com/site/advancedgisprog/other-resources/arcgis-anaconda-packages.
...
- 59
5
votes
1
answer
3k
views
Creating shapefile file with Fiona and GeoPandas?
I am trying to create a shapefile and a geojson using geopandas and fiona.
import fiona
import fiona.crs
...
geometry = [Point(xy) for xy in zip(df['longitude'],df['latitude'])]
crs = {'init': '...
- 757
5
votes
0
answers
780
views
Taking ArcGIS feature class to pandas pivot table? [closed]
I'm new to pandas and am trying to create a pivot table based off of two fields - DIVISION and MATERIAL. I'd like to count each occurances of each value. Rows would be DIVISION and columns would be ...
- 13.9k
4
votes
1
answer
12k
views
Assign a point to polygon using pandas and shapely
I have a series of Easting and Northing coordinates in pandas dataframe along with other columns of information. I know the coordinates fall within and administrative area and want to match the ...
- 1,512
4
votes
1
answer
15k
views
Mapping multiple polygons on Folium
I have a list of of coordinates that have areas mapped out on a map
Zone zone_name users distinct_users geometry
1 A 635 51 POLYGON ((13.05584728590942 77....
- 125
4
votes
2
answers
239
views
Accelerating the code for random selection of polygons
I have a shapefile with thousands of features (polygons). I need to randomly sample one polygon at every 1 sq. km. area if at least one polygon is available. I wrote the following code which works. ...
- 139
4
votes
1
answer
962
views
How to process 2 GeoDataFrames by rows pairwise?
I have 2 GeoDataFrames 100 records each, and want to calculate distances between respective geometries (Points, of course).
I can't see how to make it other than write a cycle to iterate rows ...
- 2,334
4
votes
1
answer
6k
views
Iterate over ImageCollection returning pandas dataframe using earth engine & python
I'm trying to extract values from an ImageCollection to a series of points. My goal is to create a pandas dataframe that includes the values from each image & band of the ImageCollection for each ...
- 369
4
votes
1
answer
11k
views
Parsing/converting ISO 8601 datetime fields using pandas
I have a datetime field in a Pandas dataframe that becomes problematic when using ArcGIS Pro to load data from a csv to a table on ArcSDE. The field in Pandas looks like this:
2007-02-01T05:00:00....
4
votes
1
answer
10k
views
How to extract a specific country boundary data from a GeoPandas dataset
I want to extract Singapore country boundaries from 'naturalearth_lowres' dataset from GeoPandas.
world_filepath = gpd.datasets.get_path('naturalearth_lowres')
world = gpd.read_file(world_filepath)
...
- 53
4
votes
1
answer
4k
views
GeoPandas groupby is omitting columns
I am using this answer to calculate some basic statistics of some points that fall within the bounds of a polygon (a vector grid), such that:
gridfile = 'grid.shp'
pointfile = 'points.shp'
point = ...
- 1,377
4
votes
2
answers
3k
views
Join Pandas Dataframe to Shapefile?
I have a shapefile:
Street1 Street2 Street3
Field Clark Taylor
Field Jackson State
Franklin 23rd 25th
And a Dataframe df:
Page Entry_Date
12 08012018
17 ...
- 346
4
votes
1
answer
13k
views
How to measure distance between 2 GPS points in Pandas?
I have two data frames both containing x,y GPS coordinates and I'm trying to determine if they are within a 1/4, 1/2 and 1 mile of each other. Like this:
# COORDS1
DAY LATITUDE ...
- 143
4
votes
1
answer
7k
views
Fiona + Shapely: Loading a set of LineStrings and writing their centroids to a shapefile, including original properties
I am using Fiona to load a set of links. Then I put every record in the rows of a pandas.DataFrame. Then I get the centroid of each LineString using Shapely. Afterwards I copy the schema of the ...
- 153
4
votes
1
answer
991
views
Return Geometry in a Pandas Dataframe Google Earth Engine
I have found some sample code that returns features from a GEE image to a pandas dataframe based on specific points in my feature collection.
The challenge is I need the geometry on these points to be ...
4
votes
1
answer
948
views
Setting up ArcPy virtual environment with Pandas?
Do I need to bounce back and forth between two virtual environments in order to run Pandas and ArcPy in the same script, while ensuring Pandas (or accidental updates to NumPy) doesn't break Arcpy?
I'...
- 950
4
votes
1
answer
7k
views
How to create a raster from csv points file [duplicate]
I have a CSV points file, with NDVI values.
My goal is to create a GeoTiff with 10 meters pixel, where each pixel value is calculated as the mean of points values in this square.
I read the CSV with ...
- 1,536
4
votes
1
answer
3k
views
Trying to convert GeoPandas Dataframe with date to ESRI Shapefile
Created a GeoPandas dataframe from an excel sheet. The schema for the dataframe is:
{'geometry': 'Point',
'properties': OrderedDict([
...
('DISTID', 'int'),
('...
- 43
4
votes
2
answers
2k
views
Python: Open a (line) shapefile and calculate distance to point given as a longitude-latitude tuple
I have a Pandas dataframe in Python with longitude and latitude values of certain locations (see sample below):
Index Long Lat
0 -98.58 ...
- 41