Skip to main content

Questions tagged [memory]

memory is the process in which information is encoded, stored, and retrieved.

Filter by
Sorted by
Tagged with
0 votes
1 answer
43 views

Re-writing FeatureLayer to memory causes ArcGIS Pro to Crash [closed]

I am working with Jupyter notebooks in ArcGIS Pro and would like to write some Feature layers to memory to speed up my operation. Currently it is taking roughly 24 hours. I have to emphasize it is ...
CCurtis's user avatar
  • 103
0 votes
0 answers
50 views

Networkx memory usage on steiner tree algorithm in QGIS

I'm trying to run a steiner tree for some points on a road using networkx by a simply script in QGIS. It's my first experience with this library. The data are shapefiles and are composed as follows: ...
Peter's user avatar
  • 31
0 votes
1 answer
2k views

GEE, Error generating chart: User memory limit exceeded

I want to make a scatter plot between a list of NDVI values and LST values. I've tried it all. I think is a lack of best practices in my code. Here's my script: var LST = ee.ImageCollection("...
yasmin.gelli's user avatar
2 votes
1 answer
197 views

PyQGIS load a geopackage layer into memory

I am developing a QGIS plugin. Part of the plugin generates some memory layers that the user interacts with. When the user is done with the project session, there is a button to save the memory ...
Mark Wilson's user avatar
0 votes
1 answer
576 views

Determining adjacency across elements in a GeoPandas GeoDataFrame

Main question What is an efficient (i.e., good balance of memory & processing time) way to compute adjacency using a GeoPandas GeoDataFrame? For my the rest of my question, I'll focus on the case ...
Felipe D.'s user avatar
  • 2,629
2 votes
0 answers
538 views

GEOSException: bad allocation when using pygeos geopandas buffer

I am running python 3.10 through Spyder with geopandas 0.12.0, geos 3.11.0 and pygeos 0.13. I am trying to buffer a shapefile containing lines. I have used this script on a similar dataset before and ...
Maarten 's user avatar
1 vote
0 answers
36 views

QField - application size and hidden data

I am collecting some data using QField in Ethiopia. Usually, I am collecting points (.shp) with photos (text field with link to the photo/attachment). I accidentally deleted the DCIM folder. However, ...
Terka Janek's user avatar
1 vote
1 answer
132 views

Deleting qgis setup.exe from download folder

I need to clean up the download folder in order to liberate space in C drive, but I have realised that the file qgis setup.exe is in this folder. Can I delete the qgis setup.exe file from the download ...
david 007's user avatar
1 vote
1 answer
381 views

How does "is_child_algorithm=True" work in processing.run in Python QGIS script?

I don't understand how "is_child_algorithm" works in processing.run. Premise: I created a script in python. This script uses a loop, iterating thousands of times over a field on a layer. I ...
Peter's user avatar
  • 31
2 votes
1 answer
454 views

Saving gdal_rasterize output to memory in Python script

I have the following Python line to execute GDAL's rasterize function on a shapefile subprocess.call("gdal_rasterize -te {txextent} {tyextent} -tr {resolution} -burn 1 {shapefile} {output_file}....
fuzzy_raster9873's user avatar
0 votes
1 answer
43 views

Configuring GeoTools to convert graphic objects to shapefiles without using filesystem?

By default, GeoTools (20.2) tries to convert the graphic objects into shapefiles on the disk. We want to skip this step and have GeoTools keep the file contents in memory (and return it as a byte ...
Dylan's user avatar
  • 11
2 votes
1 answer
396 views

Polygonize large raster to create mask gives memory error

I have a very large land cover/land use raster for South America with dimensions (y: 162565, x: 188387) at the Landsat 30m resolution. I want to use it to mask another raster for the same area at 270m ...
Caio Reis's user avatar
1 vote
0 answers
425 views

QGIS Python script crash when looping many times (memory problem?)

I'm having trouble with a part of a Python code that does many iterations (about 5000). The problem found is that the program crashes, it returns an error (sometimes memory error, other times all QGI ...
Peter's user avatar
  • 31
2 votes
0 answers
119 views

Memory efficient way of creating a large PostGIS table

I am trying to add the results of a query to PostGIS DB. My query converts a raster band into polygons. Although the original raster is quite big (~150GB, 12 bands) my below query runs reasonably fast ...
rm167's user avatar
  • 469
2 votes
0 answers
126 views

Optimizing memory usage when running r.viewshed in GRASS

Short explanation: I am using r.viewshed as part of a script. The viewshed analysis is the most costly (time) part of the script and I'm trying to optimize how quickly it runs (I need to run the ...
Lauren K.'s user avatar
0 votes
1 answer
567 views

Trying to process rasters in python and received CPLE_OutOfMemoryError error

I have a script that does some geoprocessing, that makes sense in a test run, but when I actually try to use it with my actual dataset, appears to totally overload my computer with memory needs, and I ...
LostinSpatialAnalysis's user avatar
0 votes
0 answers
288 views

gdalwarp releases memory VERY slowly

I’m trying to use gdalwarp to merge and reproject many elevation tif files (DEM). To do so I first create a VRT file from all the tif files and then convert and reproject to one big tif file. This is ...
Itay's user avatar
  • 1
0 votes
1 answer
370 views

GEE: reducing CHIRPS-SPI by region and time. user memory limit problem [closed]

I found a script that calculates the SPI index for places and times of your choice from the CHIRPS data. Off the shelf, the code generates a couple of images (and maps) for each place-time combination ...
Andre's user avatar
  • 1
2 votes
2 answers
954 views

PyQGIS 3: Cannot Pass Memory OUTPUT To Next Processing Algorithm As INPUT

I am attempting to convert some ArcGIS arcpy command-line scripts to PyQGIS and am running into a problem working with memory layers. The script takes several vector files as input and progresses ...
VicTorn's user avatar
  • 31
1 vote
1 answer
673 views

Rasterio merge to memory file not taking profile

I am merging datasets to a memory file using rasterio. However I can't get the memory file to take on the profile attributes passed in as dst_kwds. I get the desired result when I write the dataset to ...
g0atm1lk's user avatar
8 votes
2 answers
2k views

Adding temporary output of processing algorithm to the map canvas using PyQGIS

Let's say I run a simple algorithm like this import processing # https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/vectoranalysis.html?highlight=count%20points%20polygon#id3 ...
four-eyes's user avatar
  • 3,398
6 votes
2 answers
1k views

Why can R not allocate enough memory even for a small raster file?

I have a raster layer of vegetation cover (range from 1 to 3). The size of the file totals 60 Mb so it is not big at all. I know this is duplicating other questions, but they have not helped me out. I ...
Thomas's user avatar
  • 925
1 vote
1 answer
124 views

Why do I get memory problems reducing over specific regions?

The code below works for most countries but there are memory problems for some (e.g. Egypt, Ghana, Angola), even though these countries do not have a bigger population or land area than some countries ...
Emma's user avatar
  • 61
-1 votes
1 answer
67 views

Memory error when exporting task

I'm trying to get a building density map per square km of Africa. The following code will run ~15 hours and then report a memory error. How can I avoid this? var buildings = ee.FeatureCollection('...
Emma's user avatar
  • 61
0 votes
0 answers
225 views

gdaladdo memory issue:

I have a .gpkg raster that holds data (only) in zoom level 15. I then tried to build overviews with gdaladdo for this raster, using the following command: gdaladdo --config GDAL_CACHEMAX 1500 --config ...
rolory's user avatar
  • 101
4 votes
2 answers
3k views

Error: Execution failed; out of memory. Google Earth Engine Fails to Export Image to Asset

I am writing one of my first scripts in GEE. I have grabbed various code chucks from stack exchange and have created a memory intensive script. My goal is two take the union of two subsets of global ...
Rachel Thoms's user avatar
1 vote
0 answers
111 views

Large .las file processing in QGIS

I have a .las file that contains 100.000.000 points (2.6 Gb) and I process it by the custom QGIS plugin based on PDAL and GDAL. The problem is in processing large files as described in the previous ...
Марина Лисниченко's user avatar
0 votes
0 answers
52 views

How to use disk files to replace memfile?

I am trying to collect Sentinel images and compute their NDVI which is done quite simply. However, I need to check determine all shapely multipolygons and map a NDVI value for each. I've decided to ...
Jean-michel Lao's user avatar
2 votes
1 answer
880 views

Computation timed out when exporting time series for a large number of locations from Google Earth Engine

I am trying to export Sentinel-2 NDVI time series for a huge number of fields (n>80000) to a csv file. However I got a 'computation timed out' error after 12 hours of running. Here is the code and ...
Xinyan's user avatar
  • 21
1 vote
0 answers
114 views

QGIS file went missing?

I was working on a file for a few weeks, saving it on an external drive, in a folder, along with all layers and shapefiles. Today I went to open it and the file is gone. Not in recycle bin or anywhere ...
Mark's user avatar
  • 103
0 votes
0 answers
341 views

How much process memory is needed for ogr2ogr to process a GeoJSON file? -

I am trying to evaluate how many resources ogr2ogr needs to process a GeoJSON file; there's a concern that it's using much more memory than we would expect. All we are doing is converting a GeoJSON ...
Alan's user avatar
  • 11
2 votes
2 answers
252 views

Error generating chart: User memory limit exceeded, when changing dataset from T1_TOA to T1_SR

So, I got the code to work for T1_TOA band, but then when I switched it over to T1_SR band it gave me this error: User memory limit exceeded. So, I poked around the forum and found the code to limit ...
WhitmanCD's user avatar
6 votes
1 answer
600 views

Memory issues projecting a large raster mosaic of USGS DEMs in R

I am trying to project a large raster to UTM (meters), but I keep crashing my laptop (16GB) due to a memory shortage. It seems like using projectRaster tends to deplete the machine's memory, and some ...
Pablo Herreros Cantis's user avatar
1 vote
1 answer
270 views

QGIS Hot Spot analysis plugin Python memory error

I try to create an Hotspot crime map of Greater London. I created grid, counted points layers. When I try to use Hotspot analysis plugin after a while my memory spikes like crazy (16gigs) and this ...
bluesharxx's user avatar
1 vote
0 answers
91 views

QGIS / "create grid" consumes far more memory on linux than on windows

I am using QGIS 3.10 both on windows and on Linux with the same RAM and CPU resources. When running the tool "create grid" QGIS behaves strange on Linux: For a certain extent (~ 10x10km), ...
wookiee's user avatar
  • 11
1 vote
1 answer
729 views

Why do I have such a high memory usage for this rioxarray code?

For my code using rioxarray I am opening quite an amount of raster files. One after another. I even added del to remove those from memory, even though I thought it would not be necessary, because they ...
tetje's user avatar
  • 99
2 votes
0 answers
102 views

Releasing ArcMap memory by removing *.sr.lock files from shapefiles

Is there a method of removing *.sr.lock files from shapefiles to release memory consumption without closing out of an ArcMap application? I am attempting to run code that will cycle through 200+ ...
Weequay's user avatar
  • 101
1 vote
1 answer
667 views

How to fix the problem of 'out of memory' in PostGIS Raster processes?

I am trying to combine a series of overlapping rasters to find out the most recent data. I think I may have 10,000 layers. When I run this query: CREATE TABLE most_recent AS ( SELECT ST_Union(rast,...
milad's user avatar
  • 719
2 votes
1 answer
6k views

Read and then write rasterio GeoTIFF file without loading all data into memory

I work with large drone image raster files, on the order of 40,000 x 40,000 and above. I have a large uncompressed GeoTIFF file and I want to use rasterio to rewrite the file in compressed format. I ...
krishnab's user avatar
  • 919
3 votes
0 answers
246 views

Getting the results of a PostGIS query to a file with ogr2ogr with low memory usage

I have a PostGIS query returning geographies and attributes. The result set is deterministic and ordered (so OFFSET and LIMIT work for getting subsets). The result set of my query would exceed my RAM ...
inc42's user avatar
  • 810
0 votes
0 answers
842 views

Working around "Unable to allocate memory" error during for-loop in Arcpy

I'm running a very long for-loop in Arcpy (with ArcGIS 10.7 on Win10) over several ten thousand files. It runs smoothly until, everytime after about 3000 iterations (3-4h), it crashes with the error ...
Florian Mlehliv's user avatar
0 votes
1 answer
417 views

ArcObjects C# memory management - Out of memory exception

I'm a noob in C# and ArcObjects (started two weeks ago). I'm trying to write a stand-alone console program that will change label placement quality in all data frames for all MXD files in a given ...
user1711690's user avatar
5 votes
2 answers
85 views

Do Navigation Apps Shrink the Hippocampus?

A study shows London cab drivers' hippocampi grow compared to a control group of similar non-drivers. I presume these cab drivers don't rely on navigation apps. ...acquisition of an internal ...
Kirk Kuykendall's user avatar
1 vote
1 answer
263 views

Running into memory issues when loading a composer layout

I'm trying to load a specific composer layout model, but it not respond properly, slowing down my computer, consuming all it's memory, to the the point of giving me no option but a Windows hard reset. ...
andregeorisk's user avatar
1 vote
0 answers
233 views

pyqgis 2.18.16 - Segmentation fault error at 2nd run

When use pyqgis+python for auto-mapping, I encounter a tricky Segmentation fault (core dumped) error, at the 2nd run of the program. I checked previous similar post but not able to fix the issue. I ...
Chui's user avatar
  • 11
1 vote
0 answers
39 views

Memory management in offline routing with n waypoints?

I am trying to build an offline routing system with a map database on a portable Android device. Since I am still doing some research, I am stuck on what I think is a fundamental problem. To find the ...
user134132523's user avatar
0 votes
0 answers
62 views

ArcGIS Modelbuilder model clogs up memory?

I have a model in which a zonal histogram is computed of each line in a feature class, showing the amount of cells each line intersects in a raster file. After this, this column is joined to a table ...
Maarten 's user avatar
0 votes
1 answer
246 views

How to free the memory of return value of OGRGeometry::Buffer()?

I have in C++ (gdal 2.3.2) OGRPolygon poly; OGRGeometry* geom = poly.Buffer(1e-12, 1); now how do I free the memory allocated for geom after done using it?
hovnatan's user avatar
  • 103
0 votes
1 answer
467 views

Using Hyperion with Google Earth Engine [closed]

I am trying to use Hyperion Data for my project, unfortunately even loading the Image Collection of Hyperion exceeds my maximum 2.5 GB Data Storage. Hyperion has a revisitrate of 200 Days, so a one ...
Alexander Vocaet's user avatar
0 votes
1 answer
154 views

Pros and cons of saving data on a memory stick [closed]

I have just about reached the capacity of my laptops internal Solid State Hard Drive (Sandisk SD8SNAT-256G-1006) and I looking for some additional storage without having to buy a new machine. You ...
GISowl's user avatar
  • 11