Questions tagged [pandas]
pandas is an open source, BSD-licensed library providing high-performance, easy-to-use data structures and data analysis tools for the Python programming language.
257
questions
1
vote
1
answer
47
views
Create multiband raster in Python from X, Y
How do you create a multiband raster in Python and save it as a .tif file from a long dataframe format?
import pandas as pd
import numpy as np
from rasterio.crs import CRS
import rasterio
from ...
- 245
0
votes
0
answers
18
views
Geopy with Pandas cannot take contraints for a region
The components feature works when we geocode using a single line of code
geolocator.geocode(query='AB Vilasam HSS Muhamma',
exactly_one=True,
bounds=None, region="IN",
...
- 1
0
votes
1
answer
93
views
Grouping Polygons by Attributes and by Distance Using GeoPandas and networkx
I am working on creating a unique group ID for a parcel dataset I have containing 115k records. I have been attempting to do this with PostGIS (Grouping Parcels with an ID based on conditions, Query ...
- 27
0
votes
0
answers
76
views
ArcPy Pandas export spatial data frame to feature class
I am successfully creating two data frames with ArcPy and Pandas, 1) from an enterprise geodatabase feature class and 2) from an enterprise geodatabase table, then merging them into 3) data frame that ...
0
votes
0
answers
36
views
Spatial join of multiple GDFs and extract values
I have several different polygon geodataframes:
I would like to get the interseting area of one to another. I was able to do it like that:
frames = [oranzove, modre, cervene, tyrkysove]
empty = []
...
- 121
0
votes
0
answers
40
views
Using pandas with ArcPy, ArcGIS Pro, ArcGIS Portal and Survey123
My organization has a Python script that essentially logs into our organization’s Portal account, reads the data submitted on a corresponding Survey123 form, then creates two output tables summarizing ...
- 1
1
vote
2
answers
322
views
Create pandas DataFrame from raster image - one row per pixel with bands as columns, preserving coordinates [duplicate]
I have a 37 band image that I hoping to transform so that each row is a pixel and each column a band. I am following what was suggested in this post: Create pandas DataFrame from raster image - one ...
- 43
0
votes
0
answers
16
views
Interpolating TROPOMI Daily SIF_743 and clipping my AOI for time series in Python for my CSV file
I have a csv file that contains extracted TROPOMI SIF L2B variables (delta-time,SIF_743, lat, lon and my calculated Daily SIF_743). Now i want to extract the Daily SIF 743 and plot for lat 31.875,33....
- 21
2
votes
1
answer
261
views
Converting list of datapoints with variable CRS to WGS 84 using Python and GeoPandas
I have a file, metadata.csv, containing a list of N sensors, and their location:
sensor_id, pos_east, pos_north, crs
sensor1, 109435.44, 1157776.64, 5110
sensor2, 254194, 6593776, 25833
...
- 33
1
vote
0
answers
471
views
How to make folium multiple markers's popups display simultaneously on click on any one of them?
this code works fine but it only displays the popup of the clicked marker. I rather want that all markers' popups display when clicking on any marker of them.
import folium
import pandas as pd
data = ...
- 93
1
vote
2
answers
546
views
Building a GeoDataFrame out of a shapely GeometryCollection with shapely 2.0.0 -> ValueError: DataFrame constructor not properly called!
I successfully used to build a GeoDataFrame out of a shapely GeometryCollection (hereafter gc) as gdf = gpd.GeoDataFrame(gc) but I'm now facing this error with shapely 2.0.0:
gdf = gpd.GeoDataFrame(gc)...
- 9,546
5
votes
1
answer
527
views
Converting Pandas column with featurecollection to GeoJSON
I downloaded a CSV that contains a column which has a GeoJSON format and imported it as Pandas DataFrame. How can I convert this to a GeoJSON .geojson? I have about 10,000 rows, each with information ...
- 215
4
votes
2
answers
2k
views
Read a GDB table into a pandas dataframe in ArcPy
I have a set of tables that I have exported to a file geodatabase. I would like to read it into python as a pandas dataframe. When I try to read the file directly using pd.read_csv
import pandas as pd
...
- 417
0
votes
1
answer
30
views
How to change eps float value to geographic meter distance
I have a problem in making distance meter parameters for the DBSCAN tool. The problem is that I don't know the eps unit to use and how to change it to meters to be a parameter. Here's the Python ...
1
vote
1
answer
412
views
Why are some of my markers not showing up in Folium?
I'm creating a map in Folium and it is working great, except several hundred markers aren't showing up on the map. There is no error message, I just notice that there are fewer markers than I expect.
...
- 11
1
vote
2
answers
372
views
GeoPandas .explode() FutureWarning: Currently, index_parts defaults to True, but in the future, it will default to False to be consistent with Pandas
I have a simple GeoDataFrame containing 2 records, with MultiPolygonZ having several single parts as geometry elements. When I try to split those in their single parts using the .explode() method, it ...
- 9,546
1
vote
1
answer
341
views
(Geo)DataFrame left join leads to NaN in all right values
Let these two CSV files on the disk:
$ cat /path/to/gdf0.csv
myid,geometry
332,"MULTIPOLYGON Z (((0 0 0, 0 0 1, 0 1 1, 0 0 0)))"
220,"MULTIPOLYGON Z (((1 1 1, 1 1 2, 1 2 2, 1 1 1)))&...
- 9,546
2
votes
3
answers
1k
views
Creating GeoTIFF from Pandas DataFrame
I have a Pandas DataFrame that looks like this:
SIF_743,latitude,longitude
0.7626782059669495,-85.90831756591797,22.40107536315918
0.3139636516571045,-85.9532470703125,22.630943298339844
0....
- 149
1
vote
1
answer
414
views
Merging points to linestrings using GeoPandas
I have two GeoPandas DataFrames:
points (long,lat) : 60,000 rows
Jams segment (LineStrings) : 204,000 rows
I am trying to create a new DataFrame with the points that are located on LineStrings.
I ...
- 11
1
vote
1
answer
156
views
Make XY Event Layer in arcpy using a pandas dataframe
I have a Pandas df with Latitude and Longitude columns I am trying to export to a featureclass using ArcPy. I cannot resolve the fiona and GeoPandas issue so I tried to use MakeXYEventLayer_management ...
- 1,433
-2
votes
1
answer
836
views
Getting the intersection of two raster files using Pandas Python
I am trying to get the intersection (the overlap part) of two rasters but the thing is that by comparing them in a dataframe, I cannot find common points. I check on QGIS and there are common areas. ...
- 47
1
vote
1
answer
76
views
How to aggregate data according to week and get centroid of all points from that week?
I have a dataframe that looks like this:
I want to extract data by aggregating the dates by week, and taking a sum of the observation counts like this:
pd.DataFrame(data.groupby([pd.Grouper(key='...
- 11
0
votes
1
answer
136
views
Folium: ValueError: key_on `'FIPS'` not found
I'm working on a Folium choropleth map, using a shape file from the Census bureau and religion data from the US Religion Census. I can get both sets of data loaded and cleaned up, so I have two DFs:
...
- 33
1
vote
0
answers
146
views
Recreating QGIS SAGA polygon self-intersection in python? [closed]
so I am trying to find a way to re-create the SAGA polygon self-intersection tool used in QGIS in python, but I am having some trouble figuring out how to organize the workflow. with the SAGA polygon ...
- 63
0
votes
1
answer
48
views
Creating an arcTool from Python Script - error with the output
I want to add this script to an ArcToolbox for a coworker, so that they can reorder a table's columns after exporting a table to excel. They are not savvy with code, so I'm trying to just provide them ...
- 1
2
votes
1
answer
11k
views
ValueError: Columns must be same length as key
I want to geocode a number of universities in Europe (stored in a csv file). The csv file contains only one column (Name). However, I am yet to complete the work, since my code contains bugs. When I ...
- 89
0
votes
1
answer
42
views
How can I fill a feature layer with data from an xlsx\xls spread sheet if the field names are different (mapping)
I'm developing a widget that suppose to take data from a specific spread sheet of an excel file (xlsx or xls) and fill a layer that I created using arcmap pro.. The layer has slightly different field ...
2
votes
5
answers
2k
views
Converting Pandas DataFrame to point feature dataset via ArcGIS function (e.g. XY Table To Point) in ArcPy
I have a specified output path output_path, and a Pandas DataFrame df with two rows containing geoinformation (i.e. x and y coordinates):
#import relevant module
import pandas as pd
#define output ...
- 147
0
votes
1
answer
1k
views
Getting area (%) of two intersecting polygons in Python
I am trying to calculate the area of two intersecting polygons in my GDB using Python (probably geopandas).
I have found solution using PostGIS:
SELECT layer.text,
layer_2.text,
sum(...
- 121
0
votes
1
answer
96
views
GeoPandas in ArcGIS Pro not updating operation performed
I am new in the programming area. I have done some code making use of sources on the web. That is why I am having trouble. In my query I have used GeoPandas, Pandas and numpy library to make ...
0
votes
0
answers
56
views
nlargest distance between point and other points referenced as list of ids
I have the geodataframe which has point geometry and a list of 'children' ids related to that point (could be empty or any number or children)
import geopandas as gpd
import pandas as pd
from shapely ...
- 335
0
votes
0
answers
52
views
Updating ArcGIS feature class from Python script not cooperating to code
I’m trying to update the Project_Name field in my feature class table with the following code:
import arcpy
import csv
import pandas as pd
with open(r'thepath.csv') as infile:
...
0
votes
0
answers
97
views
Geometry calculation using Pandas UDF
I am creating Pandas UDF to calculate the Geometry length .The following is the code which I created to calculate Geometry length.
from pyspark.sql.functions import pandas_udf
import pandas as pd
from ...
- 101
1
vote
1
answer
2k
views
Merge several JSON files into one CSV file
I have about 37 nested JSON files. The keys and the values are same in all the files. My objective is to combine the data in all the files into a single CSV file so I can analyze the output . I have ...
- 89
1
vote
1
answer
249
views
Create a shapefile of concentric circles in Python by buffering a point at different levels
The following code returns only the last circle's geometry. I need a shapefile containing geometry of all buffer operations.
import pandas as pd
import geopandas as gpd
pd.options.mode....
- 19
0
votes
0
answers
262
views
Google Earth Engine: loop raster extraction over image collection and create dataframes (geemap)
I would like to extract a large number of points (approximately 3000) from each image within an image collection, and return a pandas dataframe for each image.
The function I'm using to extract raster ...
- 41
1
vote
1
answer
942
views
Merge two GeoJSON into one in a dataframe
I have a dataframe containing GeoJSON:
geojson checkpoint lom_name
{"type":"LineString","coordinates":[[1,4],[2,5]]} 6 ...
- 41
2
votes
1
answer
322
views
Using Select by Attributes for large list of values in ArcPy
I have a CSV of integers containing meter numbers varying from 9 digits to 11 digits. I've written a simple script to read CSV to a list.
import pandas as pd
path = "example_path.csv"
df = ...
- 23
1
vote
0
answers
149
views
Issue calling a subprocess from ArcGIS Pro Toolbox
I am developing a toolbox that - at some point - has to process a large amount of data and apply a function to each row. For efficiency reasons I already use a pandas dataframe.
I am familiar with ...
0
votes
1
answer
235
views
Create a geopackage file with integer attribute containing NULL values in GeoPandas
I've been asked to produce a geopackage file in a specified format where one attribute is specified to be int64 nullable, and my case includes NULL values. Trying in GeoPandas, my problem is that I ...
- 1
0
votes
1
answer
194
views
Geodataframe to SHP in loop and name of the GDF
I am trying to export multiple gdf to shp and name these shp by the gdf name.
I have found this solution
here is my list:
maloobchod_list = [maloobchod2006_geom, maloobchod2011_geom, ...
- 121
0
votes
2
answers
168
views
Clip data using GeoPandas or PostGIS [closed]
I have two geodataframes - points. First gdf contains smaller number of points. I am trying to clip my second gdf (with higher number of points and therefore bigger area) to the extent of first gdf. ...
- 121
0
votes
0
answers
60
views
Cannot read SHP polygon using Geopandas [duplicate]
I am trying to read ESRI SHP Polygon using geopandas:
polygon=gpd.read_file(r'Path.shp', encoding='utf-8', driver='ESRI Shapefile')
But I am getting this error:
NotImplementedError: A polygon does ...
- 121
3
votes
1
answer
543
views
Best way to check if a point is in a region?
I'm working with Chicago PD crash data that includes Lat/Long. I've read the crashes data into the crashes DF.
It doesn't include the city ward number, so I'd like to add it. I figured out how to do ...
- 33
-1
votes
2
answers
482
views
Pandas how to convert two columns to many columns and rows based on mutual row to row relationship [closed]
Let's say we have a df with row to row relation of some IDs, for example like this:
df = pd.DataFrame({"ID_nm": [1, 2, 4, 2, 3, 4, 5, 1, 6],
"ID_lt": ["a&...
- 183
0
votes
0
answers
605
views
Nominatim does not provide zipcode for some latitude/longitude values
I am using Nominatim from the GeoPy Python library to obtain Zip Code given Latitude and Longitude.
def get_zip_code(x):
geolocator = geopy.Nominatim(user_agent="check_1")
location =...
- 1
2
votes
1
answer
486
views
How to create geometry column from nested lists
How can I create mixed geometries types from nested lists ?
from pandas import DataFrame
row1 = [
[
[
[-73.932204, 45.491553], [-73.931682, 45.491085],
[...
- 209
1
vote
0
answers
123
views
Efficiently calculating the total time a cell is occupied by comparing the line segments within a cell and their speed [closed]
I have a 1km square grid and what I want to do find the total length of all line segment within that cell and multiply it by each individual lines speed variable to get a total time occupied value to ...
- 1,433
1
vote
1
answer
101
views
Calculating time fields in Pandas\Geopandas
I have a DataFrame that consists of 2 datetime fields I am trying to calculate meters a min so I thought I could use a definition and the df['newfield'] = something method. So I created a definition ...
- 1,433
2
votes
1
answer
2k
views
WKTReadingError: Could not create geometry because of errors while reading input
I have a pandas DataFrame that I am trying to convert to geopandas. I tried a few different methods to convert and also attempted to fix the invalid geometries. Code below:
import pandas as pd
import ...
- 465