Questions tagged [memory]

memory is the process in which information is encoded, stored, and retrieved.

Filter by
Sorted by
Tagged with
0
votes
0answers
21 views

gdalwarp releases memory VERY slowly

I’m trying to use gdalwarp to merge and reproject many elevation tif files (DEM). To do so I first create a VRT file from all the tif files and then convert and reproject to one big tif file. This is ...
0
votes
1answer
29 views

GEE: reducing CHIRPS-SPI by region and time. user memory limit problem [closed]

I found a script that calculates the SPI index for places and times of your choice from the CHIRPS data. Off the shelf, the code generates a couple of images (and maps) for each place-time combination ...
2
votes
2answers
55 views

PyQGIS 3: Cannot Pass Memory OUTPUT To Next Processing Algorithm As INPUT

I am attempting to convert some ArcGIS arcpy command-line scripts to PyQGIS and am running into a problem working with memory layers. The script takes several vector files as input and progresses ...
0
votes
0answers
57 views

Rasterio merge to memory file not taking profile

I am merging datasets to a memory file using rasterio. However I can't get the memory file to take on the profile attributes passed in as dst_kwds. I get the desired result when I write the dataset to ...
5
votes
2answers
120 views

Adding temporary output of processing algorithm to the map canvas using PyQGIS

Let's say I run a simple algorithm like this import processing # https://docs.qgis.org/3.16/en/docs/user_manual/processing_algs/qgis/vectoranalysis.html?highlight=count%20points%20polygon#id3 ...
4
votes
2answers
254 views

Why can R not allocate enough memory even for a small raster file?

I have a raster layer of vegetation cover (range from 1 to 3). The size of the file totals 60 Mb so it is not big at all. I know this is duplicating other questions, but they have not helped me out. I ...
1
vote
1answer
35 views

Why do I get memory problems reducing over specific regions?

The code below works for most countries but there are memory problems for some (e.g. Egypt, Ghana, Angola), even though these countries do not have a bigger population or land area than some countries ...
-1
votes
1answer
27 views

Memory error when exporting task

I'm trying to get a building density map per square km of Africa. The following code will run ~15 hours and then report a memory error. How can I avoid this? var buildings = ee.FeatureCollection('...
0
votes
0answers
34 views

gdaladdo memory issue:

I have a .gpkg raster that holds data (only) in zoom level 15. I then tried to build overviews with gdaladdo for this raster, using the following command: gdaladdo --config GDAL_CACHEMAX 1500 --config ...
2
votes
1answer
268 views

Error: Execution failed; out of memory. Google Earth Engine Fails to Export Image to Asset

I am writing one of my first scripts in GEE. I have grabbed various code chucks from stack exchange and have created a memory intensive script. My goal is two take the union of two subsets of global ...
1
vote
0answers
47 views

Large .las file processing in QGIS

I have a .las file that contains 100.000.000 points (2.6 Gb) and I process it by the custom QGIS plugin based on PDAL and GDAL. The problem is in processing large files as described in the previous ...
0
votes
0answers
13 views

Memory error when using LecoS vector overlay

I am currently trying to use the overlay raster metrics of LecoS. But am experiencing some difficulties. I do have the following error code: File "C:\PROGRA~1\QGIS3~1.16\apps\Python37\lib\site-...
0
votes
0answers
31 views

How to use disk files to replace memfile?

I am trying to collect Sentinel images and compute their NDVI which is done quite simply. However, I need to check determine all shapely multipolygons and map a NDVI value for each. I've decided to ...
1
vote
0answers
113 views

Computation timed out when exporting time series for a large number of locations from Google Earth Engine

I am trying to export Sentinel-2 NDVI time series for a huge number of fields (n>80000) to a csv file. However I got a 'computation timed out' error after 12 hours of running. Here is the code and ...
0
votes
0answers
82 views

Mosaicing rasters Rasterio vs GDAL

I want to create a mosaic using multiple tiles. Firstly I've tried GDAL using gdalbuildvrt + gdal_translate commands. Then I wanted to use Rasterio for creating mosaics. I've used rasterio.merge.merge ...
1
vote
0answers
29 views

QGIS file went missing?

I was working on a file for a few weeks, saving it on an external drive, in a folder, along with all layers and shapefiles. Today I went to open it and the file is gone. Not in recycle bin or anywhere ...
0
votes
0answers
67 views

How much process memory is needed for ogr2ogr to process a GeoJSON file? -

I am trying to evaluate how many resources ogr2ogr needs to process a GeoJSON file; there's a concern that it's using much more memory than we would expect. All we are doing is converting a GeoJSON ...
2
votes
2answers
65 views

Error generating chart: User memory limit exceeded, when changing dataset from T1_TOA to T1_SR

So, I got the code to work for T1_TOA band, but then when I switched it over to T1_SR band it gave me this error: User memory limit exceeded. So, I poked around the forum and found the code to limit ...
4
votes
1answer
167 views

Memory issues projecting a large raster mosaic of USGS DEMs in R

I am trying to project a large raster to UTM (meters), but I keep crashing my laptop (16GB) due to a memory shortage. It seems like using projectRaster tends to deplete the machine's memory, and some ...
1
vote
1answer
112 views

QGIS Hot Spot analysis plugin Python memory error

I try to create an Hotspot crime map of Greater London. I created grid, counted points layers. When I try to use Hotspot analysis plugin after a while my memory spikes like crazy (16gigs) and this ...
1
vote
0answers
36 views

QGIS / "create grid" consumes far more memory on linux than on windows

I am using QGIS 3.10 both on windows and on Linux with the same RAM and CPU resources. When running the tool "create grid" QGIS behaves strange on Linux: For a certain extent (~ 10x10km), ...
0
votes
1answer
104 views

Why do I have such a high memory usage for this rioxarray code?

For my code using rioxarray I am opening quite an amount of raster files. One after another. I even added del to remove those from memory, even though I thought it would not be necessary, because they ...
2
votes
0answers
58 views

Releasing ArcMap memory by removing *.sr.lock files from shapefiles

Is there a method of removing *.sr.lock files from shapefiles to release memory consumption without closing out of an ArcMap application? I am attempting to run code that will cycle through 200+ ...
1
vote
1answer
215 views

How to fix the problem of 'out of memory' in PostGIS Raster processes?

I am trying to combine a series of overlapping rasters to find out the most recent data. I think I may have 10,000 layers. When I run this query: CREATE TABLE most_recent AS ( SELECT ST_Union(rast,...
1
vote
1answer
2k views

Read and then write rasterio GeoTIFF file without loading all data into memory

I work with large drone image raster files, on the order of 40,000 x 40,000 and above. I have a large uncompressed GeoTIFF file and I want to use rasterio to rewrite the file in compressed format. I ...
3
votes
0answers
60 views

Getting the results of a PostGIS query to a file with ogr2ogr with low memory usage

I have a PostGIS query returning geographies and attributes. The result set is deterministic and ordered (so OFFSET and LIMIT work for getting subsets). The result set of my query would exceed my RAM ...
0
votes
0answers
214 views

Working around "Unable to allocate memory" error during for-loop in Arcpy

I'm running a very long for-loop in Arcpy (with ArcGIS 10.7 on Win10) over several ten thousand files. It runs smoothly until, everytime after about 3000 iterations (3-4h), it crashes with the error ...
0
votes
1answer
163 views

ArcObjects C# memory management - Out of memory exception

I'm a noob in C# and ArcObjects (started two weeks ago). I'm trying to write a stand-alone console program that will change label placement quality in all data frames for all MXD files in a given ...
5
votes
2answers
74 views

Do Navigation Apps Shrink the Hippocampus?

A study shows London cab drivers' hippocampi grow compared to a control group of similar non-drivers. I presume these cab drivers don't rely on navigation apps. ...acquisition of an internal ...
1
vote
1answer
212 views

Running into memory issues when loading a composer layout

I'm trying to load a specific composer layout model, but it not respond properly, slowing down my computer, consuming all it's memory, to the the point of giving me no option but a Windows hard reset. ...
1
vote
0answers
83 views

pyqgis 2.18.16 - Segmentation fault error at 2nd run

When use pyqgis+python for auto-mapping, I encounter a tricky Segmentation fault (core dumped) error, at the 2nd run of the program. I checked previous similar post but not able to fix the issue. I ...
1
vote
0answers
25 views

Memory management in offline routing with n waypoints?

I am trying to build an offline routing system with a map database on a portable Android device. Since I am still doing some research, I am stuck on what I think is a fundamental problem. To find the ...
0
votes
0answers
36 views

ArcGIS Modelbuilder model clogs up memory?

I have a model in which a zonal histogram is computed of each line in a feature class, showing the amount of cells each line intersects in a raster file. After this, this column is joined to a table ...
0
votes
1answer
99 views

How to free the memory of return value of OGRGeometry::Buffer()?

I have in C++ (gdal 2.3.2) OGRPolygon poly; OGRGeometry* geom = poly.Buffer(1e-12, 1); now how do I free the memory allocated for geom after done using it?
0
votes
1answer
250 views

Using Hyperion with Google Earth Engine [closed]

I am trying to use Hyperion Data for my project, unfortunately even loading the Image Collection of Hyperion exceeds my maximum 2.5 GB Data Storage. Hyperion has a revisitrate of 200 Days, so a one ...
0
votes
1answer
63 views

Pros and cons of saving data on a memory stick [closed]

I have just about reached the capacity of my laptops internal Solid State Hard Drive (Sandisk SD8SNAT-256G-1006) and I looking for some additional storage without having to buy a new machine. You ...
0
votes
0answers
98 views

Exporting Layer to Feature Class is Incomplete without Error

I have a CSV of 1.2 million records that I want export to a GDB as a feature class. I've done this numerous times, but this was before upgrading from 10.5. Steps: Add CSV Display X/Y table Export to ...
2
votes
0answers
38 views

Using destructors in PyQGIS?

Related to my Problem described in PyQGIS memory issues, I'd like to try using destructors for QgsGeometry and other classes. I dont get how to use them though. According to https://qgis.org/api/2.18/...
3
votes
0answers
226 views

PyQGIS memory issues [closed]

I have some big problems with the memory usage of my standalone PyQgis script based on QGis 2.18 and run on ubuntu/linux (happens also on Windows 10 though). I will document it very precisely using ...
0
votes
1answer
312 views

Geoserver - how clear memory after tile-seeding and normal wmts queries

I am currently doing a lot of seeding of tiles in GeoServer and I noticed that after the tiling process is over the memory that is required during the task is not cleared. The same thing happens after ...
2
votes
1answer
56 views

Watershed issue

I am attempting to create a catchment of Danube. So far I have used the r.fill.dir to create a depressionless map and flow directions. Thus, I am now looking to use r.watershed to create stream ...
0
votes
1answer
190 views

Topology error in ArcGis

I am trying to intersect a large feature class (118692 elements) with a grid (58000 elements) but I get an error message. Traceback (most recent call last): File "Path/InterJoin.py", line 24, in &...
2
votes
1answer
1k views

How to increase Java heap size on GeoServer on Windows?

I tried setting wrapper.java.maxmemory=1300 parameter in c:\Program Files (x86)\GeoServer 2.11.1\wrapper\wrapper.conf but I can't set more than 1300MB (geoserver service can't start). I tried ...
3
votes
1answer
1k views

Creating permanent vector layer in QGIS

I am currently creating a vector layer like following: outLayer = QgsVectorLayer('Linestring?crs='+ crs, 'graph_with_azimuth_ogr' , 'memory') The outLayer is now stored in memory, which means it only ...
0
votes
0answers
217 views

Mapshaper running out of memory on R

I'm using rmapshaper, the R wrapper of MapShaper, to simplify polygons without slivers and it works well on small shapefiles. However, when I run it on a larger one (~150M, ~5000 shapes), it runs out ...
5
votes
2answers
796 views

Memory leak in ArcGIS Pro: in_memory workspace is not deleted

I have a python script for ArcGIS Pro 2.0.1 that executes the same code with the same data size in a loop. So intuitively the run times of the iterations of the loop shall be about the same. However, ...
3
votes
0answers
154 views

GeoJSON files using too much memory?

I have about/more than 400 GeoJSON layer files in my QGIS project. When I checked the task manager memory, it displays qgis.bin.exe *32 using about 1,334,000 of CPU memory. I had another version ...
5
votes
1answer
1k views

Insufficient memory for QGIS Raster Calculator?

I read related posts, but could not find an answer. I'm trying to perform an operation on two "light" rasters (80mg and 15mg), but get this error over and over. Any suggestion? This operation worked ...
3
votes
2answers
1k views

Error in raster Analysis in QGIS

I'm trying to conduct raster analysis in QGIS on a digital elevation model with a grid resolution of 5 cm. The disk size for the data set is 151 MB. When using default GDAL raster calculator it throws ...
1
vote
1answer
430 views

Duplicating memory layers in QGIS

I kept my original question clear and concise to make it easier to answer. This is a response to requests for more information. Introduction I am using QGIS version 2.18.9. I created about 20 new ...