1

I am working on a 13407 lines * 11050 columns raster file (SRTM DEM, 30m) with a script that calls a series of hydrological modules in GRASS 7.0. The system I am using is a Windows server 2012 R2, 64 bit Intel Xeon 2.5 GHz processor, 32GB of RAM.

The first module called in the script is r.watershed. When launched on that data set, it generates the following error : G_malloc: impossible to allocate 1185910608 bytes to raster/r.watershed/ram/init_vars.c:149

I managed to pass that first hurdle using the -m flag on r.watershed. The scripts now requires 7 GB of disk space, even if it's slow and I don't understand why it can't use the ram in the first place, r.watershed finishes correctly.

However, when the script gets to the next module, r.fill.dir, GRASS crashes at 'Reading input elevation raster map...' with the following message:

Problem signature:
  Problem Event Name:   APPCRASH
  Application Name: r.fill.dir.EXE
  Application Version:  7.0.0.0
  Application Timestamp:    54e7aaf0
  Fault Module Name:    r.fill.dir.EXE
  Fault Module Version: 7.0.0.0
  Fault Module Timestamp:   54e7aaf0
  Exception Code:   c00000fd
  Exception Offset: 00001479
  OS Version:   6.3.9600.2.0.0.400.8
  Locale ID:    1036
  Additional Information 1: 3397
  Additional Information 2: 33978ebc9cf04f9490066e458551979d
  Additional Information 3: caf7
  Additional Information 4: caf72cf53bf264b7d49f3bd72b0afc04

The script runs fine on a smaller data set.

Any idea of how to solve the app crash?

Edit: as requested, here is the result of g.region -p:

g.region -p                                                            projection: 1 (UTM)
zone:       22
datum:      wgs84
ellipsoid:  wgs84
north:      636094.00956457
south:      233878.203125
west:       98924.52245413
east:       430413.65625
nsres:      30.00043309
ewres:      29.99901663
rows:       13407
cols:       11050
cells:      148147350
  • 2
    Welcome to GIS SE. Please take the Tour to better understand how we do things here. It is a policy that each Question should ask only one question, and you've got two very different questions. Since the memory limitations of 32-bit applications are a general computing question, not specific to GIS, I'd recommend that you edit your question to focus on the second issue. – Vince Sep 9 '15 at 20:13
  • You'll need to provide a lot more information about the exact command directive and more details about the associated data file(s), and you'll probably still run into the fact that GIS SE is not intended to replace GIS software provider's' Tech Support solutions. Even if you were to use a 64-bit application, processing 1-8Gb images is a less than ideal situation. – Vince Sep 10 '15 at 18:11
  • Please edit the posting and add the output of "g.region -p" - likely your current grasswiki.osgeo.org/wiki/Computational_region is not matching the elevation map, i.e. you try to calculate at a much higher resolution than needed (just guessing, but a common mistake)... – markusN Sep 10 '15 at 19:12
3

Very likely your computational region is not matching the elevation map, i.e. too large. Check it with

g.region -p

I just used the EU DEM 25m to make a test on my tiny ASUS laptop (4GB RAM, Intel i3), using Fedora 22, 64bit:

13407 * 11050 = 148147350 <<--- your DEM

12880 * 16370 = 210845600 <<-- my DEM

(I just had this DEM ready here to play with)

GRASS 7.1.svn (eu_laea):~ > g.region -p
projection: 99 (Lambert Azimuthal Equal Area)
zone:       0
datum:      etrs89
ellipsoid:  grs80
north:      2699750
south:      2377750
west:       4126750
east:       4536000
nsres:      25
ewres:      25
rows:       12880
cols:       16370
cells:      210845600

Results:

RAM: it used some swap memory since I have browser etc open at the same time.

[neteler@oboe ~]$ free
              total        used        free      shared  buff/cache   available
Mem:        3930508     3600648       32408       93164      297452      165584
Swap:       3932156     2735164     1196992

Timing:

GRASS 7.1.svn (eu_laea):~ > time -p r.watershed elevation=eu_dem_25_TN accumulation=eu_dem_25_TN.acc basin=eu_dem_25_TN.watersheds threshold=10000
SECTION 1a (of 5): Initiating Memory.
SECTION 1b (of 5): Determining Offmap Flow.
 100%
SECTION 2: A* Search.
 100%
SECTION 3a: Accumulating Surface Flow with MFD.
 100%
SECTION 3b: Adjusting drainage directions.
 100%
SECTION 4: Watershed determination.
 100%
SECTION 5: Closing Maps.

real 1270.30
user 1048.68
sys 54.00

... 21 minutes.

Given my test on a small laptop, it is very likely that your actual computational region does not match the input map.

Concerning r.fill.dir: Why do you use the r.fill.dir module?

  • I define the computational area with 'g.region raster=dem zoom=dem' ('dem' being my raster) before lunching the script so it should match the elevation map. r.fill.dir is to extract the depression areas as sinks. – fred Sep 10 '15 at 20:36
  • ok - Since you use zoom=dem please add also res=xx (the elevation map's raster resolution) to the g.region command along with the -a flag to be sure to enforce that resolution (in order to avoid non-square pixels as you report here: lists.osgeo.org/pipermail/grass-user/2015-September/073009.html) – markusN Sep 11 '15 at 15:28
  • I have tried these commands on the region but still have the same crash when it gets to r.fill.dir. Also tried on different systems with similar sized data sets with the same result. Does it work on your data set? – fred Sep 14 '15 at 14:15
  • OK, I tried it myself now and indeed, got a crash (happens only with larger maps it seems). I opened a bug report for that: trac.osgeo.org/grass/ticket/2742 - Meanwhile you may check the addon grass.osgeo.org/grass70/manuals/addons/r.hydrodem.html (install with the extension manager in the user interface under "Settings"). The filled sinks I believe you can get with map algebra (r.mapcalc) of original DEM minus the output of r.hydrodem. – markusN Sep 15 '15 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.