3

Index construction is an I/O-intensive task, not generally CPU-bound. It's also likely to involve locks on table extents, and is therefore not amenable to multiprocessing (INDEX requests to a single table in multiple sessions are likely to contend with one another). Similarly, multiple INDEX requests on different tables in a single datafile are likely to ...


2

In general GRASS does not support parallelized operation. There are some workarounds: on the wiki. However for this particular question, are you familiar with the r.reclass module? Creating a reclass raster is very "cheap". I guess the question is: what are you planning to do with these 3652 binary rasters after you create them? (added after ...


1

Let's benchmark some different approaches. Here is some example data. Please note that we explode the MULTIPOLYGON geometry to simplify matters a bit. library(sf) library(raster) library(terra) library(exactextractr) nc <- st_read(system.file("shape/nc.shp", package="sf")) nc <- st_cast(nc, "POLYGON") r <- raster(...


1

Following my comment I will try to explain the steps of using tabularaster to speed up the extraction process. First let's get ourselves a vector data object and a raster: library(raster) library(tabularaster) library(sf) library(dplyr) ## Get some polygons nc <- st_read(system.file("shape/nc.shp", package="sf")) # Generate a raster ...


1

I use GNU parallel and r.mapcalc a lot. I think the issue is quotes. Try this: seq $x | parallel "r.mapcalc \"expression=tas.binary{}=if(tasmax_day_CMCC-CMS_rcp45_r1i1p1_20300101-20391231.{}<293.15,1,0)\"" OR seq $x | parallel "r.mapcalc 'expression=tas.binary{}=if(tasmax_day_CMCC-CMS_rcp45_r1i1p1_20300101-20391231.{}<293.15,1,...


1

It looks like you are working with data in a geodatabase. Multiprocessing doesn't work on feature classes in a geodatabase because each update acquires a lock. According to https://www.esri.com/arcgis-blog/products/arcgis-desktop/analytics/multiprocessing-with-arcgis-approaches-and-considerations-part-1/: ...will not work with feature classes in a file ...


1

A quick way to test if a speedup is possible is to only process some of the polygons in one run and save the result as a geojson-file. If you change your script to take a batch number and total number of batches as input, you can run it from the command line. And if you start multiple runs at the same time you test if a speedup is possible. I have written ...


Only top voted, non community-wiki answers of a minimum length are eligible