2

I want to run BFAST on a raster stack (768 Landsat NDVI) in parallel processing. In order to illustrate the problem, I used the following reproducible code:

Load necessary packages;

library(raster)
library(zoo)
library(xts)
library(bfast)

Create a reproducible rasterstack;

set.seed(123)
r1 <- raster(nrows = 6, ncols = 5, res = 30, xmn = 267195, xmx = 267375, ymn = 4016985, ymx = 4017135)
rr <- lapply(1:768, function(i) setValues(r1, runif(ncell(r1), min = -10000, max = 10000)))
ndvi <- stack(rr)
crs(ndvi) <- "+proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0"

Create date function;

dt <-as.data.frame(as.Date(c(
  '2017-01-03','2017-02-04','2018-11-22','2019-05-17','2013-04-14','2013-04-30',
  '2013-05-16','2013-06-01','2013-06-17','2013-07-03','2013-07-19','2013-08-04',
  '2013-08-20','2013-09-05','2013-10-07','2013-10-23','2013-11-08','2013-11-24',
  '2013-12-10','2013-12-26','2014-01-11','2014-01-27','2014-02-12','2014-02-28', 
  '2014-04-01','2014-04-17','2014-05-03','2014-05-19','2014-06-04','2014-06-20',
  '2014-07-06','2014-07-22','2014-08-07','2014-08-23','2014-09-08','2014-09-24',
  '2014-10-10','2014-10-26','2014-11-11','2014-11-27','2014-12-13','2014-12-29',
  '2015-01-14','2015-01-30','2015-02-15','2015-03-19','2015-04-04','2015-04-20',
  '2015-05-06','2015-05-22','2015-06-07','2015-06-23','2015-07-09','2015-07-25',
  '2015-08-10','2015-08-26','2015-09-11','2015-09-27','2015-10-13','2015-10-29',
  '2015-11-14','2015-11-30','2015-12-16','2016-01-01','2016-01-17','2016-02-02',
  '2016-02-18','2016-03-05','2016-03-21','2016-04-06','2016-04-22','2016-05-08',
  '2016-05-24','2016-06-25','2016-07-11','2016-07-27','2016-08-12','2016-08-28',
  '2016-09-13','2016-09-29','2016-10-15','2016-10-31','2016-11-16','2016-12-02',
  '2016-12-18','2017-01-19','2017-02-20','2017-03-08','2017-03-24','2017-04-09',
  '2017-04-25','2017-05-11','2017-05-27','2017-06-12','2017-06-28','2017-07-14',
  '2017-07-30','2017-08-15','2017-08-31','2017-09-16','2017-10-02','2017-10-18',
  '2017-11-03','2017-12-05','2017-12-21','2018-01-06','2018-01-22','2018-02-07',
  '2018-02-23','2018-03-11','2018-04-12','2018-04-28','2018-05-14','2018-05-30', 
  '2018-06-15','2018-07-01','2018-07-17','2018-08-02','2018-08-18','2018-09-03',
  '2018-09-19','2018-10-05','2018-10-21','2018-11-06','2018-12-08','2018-12-24',
  '2019-01-09','2019-01-25','2019-02-10','2019-02-26','2019-03-14','2019-03-30',
  '2019-04-15','2019-05-01','2019-06-02','2019-06-18','2019-07-04','2019-07-20',
  '2019-08-05','2019-08-21','2019-09-06','2019-09-22','2019-10-08','2019-10-24',
  '2019-11-09','2019-11-25','2001-01-31','2002-03-23','2002-11-02','2007-03-21',
  '2008-02-04','2010-05-16','2010-06-17','2012-01-14','2012-12-15','2013-01-16',
  '2013-05-08','2013-12-02','2014-03-24','2014-04-25','2014-05-11','2015-02-07',
  '2015-02-23','2015-08-02','2016-02-10','2016-03-13','2016-12-10','2018-04-20',
  '2018-05-06','2019-01-01','2019-07-28','1999-08-06','1999-09-07','1999-10-25',
  '1999-12-12','2000-01-29','2000-02-14','2000-03-01','2000-03-17','2000-04-18',
  '2000-05-20','2000-06-05','2000-06-21','2000-08-24','2000-09-09','2000-10-11',
  '2000-10-27','2000-11-28','2000-12-30','2001-01-15','2001-03-04','2001-03-20',
  '2001-04-21','2001-05-23','2001-06-08','2001-08-11','2001-09-28','2002-01-02',
  '2002-01-18','2002-04-08','2002-04-24','2002-06-11','2002-08-30','2002-10-01',
  '2003-03-10','2003-03-26','2003-04-27','2003-05-13','2003-05-29','2003-07-16',
  '2003-09-02','2003-09-18','2003-10-20','2004-01-24','2004-03-12','2004-03-28',
  '2004-04-13','2004-05-15','2004-06-16','2004-07-02','2004-07-18','2004-09-04',
  '2004-09-20','2004-10-22','2004-11-07','2004-11-23','2004-12-09','2004-12-25',
  '2005-01-10','2005-03-15','2005-03-31','2005-04-16','2005-05-02','2005-05-18',
  '2005-06-03','2005-06-19','2005-07-05','2005-07-21','2005-08-06','2005-08-22',
  '2005-09-07','2005-09-23','2005-10-09','2005-12-12','2006-04-03','2006-04-19',
  '2006-05-05','2006-05-21','2006-06-06','2006-06-22','2006-07-08','2006-07-24',
  '2006-08-09','2006-08-25','2006-09-10','2006-09-26','2006-11-13','2007-02-01',
  '2007-03-05','2007-04-22','2007-05-08','2007-06-09','2007-06-25','2007-07-11',
  '2007-07-27','2007-08-12','2007-08-28','2007-09-13','2007-09-29','2007-10-15',
  '2007-10-31','2007-11-16','2007-12-02','2008-01-19','2008-02-20','2008-03-23',
  '2008-04-24','2008-05-10','2008-05-26','2008-06-11','2008-06-27','2008-07-13',
  '2008-07-29','2008-08-14','2008-08-30','2008-09-15','2008-10-01','2008-10-17',
  '2008-11-18','2008-12-04','2009-01-05','2009-01-21','2009-03-10','2009-03-26',
  '2009-05-13','2009-05-29','2009-06-14','2009-06-30','2009-07-16','2009-08-01',
  '2009-08-17','2009-09-02','2009-09-18','2009-10-04','2009-10-20','2009-11-05',
  '2009-11-21','2009-12-07','2009-12-23','2010-01-08','2010-03-13','2010-04-14',
  '2010-06-01','2010-07-03','2010-07-19','2010-08-04','2010-08-20','2010-09-05',
  '2010-09-21','2010-10-07','2010-10-23','2010-11-08','2010-11-24','2010-12-10',
  '2010-12-26','2011-01-11','2011-01-27','2011-03-16','2011-05-03','2011-05-19',
  '2011-06-04','2011-06-20','2011-07-06','2011-07-22','2011-08-07','2011-08-23',
  '2011-09-08','2011-09-24','2011-10-10','2011-10-26','2011-11-11','2011-11-27',
  '2011-12-13','2011-12-29','2012-01-30','2012-02-15','2012-03-02','2012-03-18',
  '2012-04-03','2012-04-19','2012-05-05','2012-05-21','2012-06-06','2012-06-22',
  '2012-07-08','2012-07-24','2012-08-09','2012-08-25','2012-09-10','2012-09-26',
  '2012-10-12','2012-10-28','2012-11-13','2012-12-31','2013-02-01','2013-02-17',
  '2013-03-05','2013-03-21','2013-04-22','2013-05-24','2013-06-09','2013-06-25',
  '2013-07-11','2013-07-27','2013-08-12','2013-08-28','2013-09-13','2013-09-29',
  '2013-10-15','2013-10-31','2013-11-16','2013-12-18','2014-01-03','2014-01-19',
  '2014-02-04','2014-02-20','2014-03-08','2014-04-09','2014-05-27','2014-06-12',
  '2014-06-28','2014-07-14','2014-07-30','2014-08-31','2014-09-16','2014-10-02',
  '2014-10-18','2014-11-03','2014-11-19','2014-12-05','2014-12-21','2015-01-06',
  '2015-01-22','2015-03-11','2015-03-27','2015-04-12','2015-04-28','2015-05-14',
  '2015-05-30','2015-06-15','2015-07-01','2015-07-17','2015-08-18','2015-09-03',
  '2015-09-19','2015-10-05','2015-10-21','2015-11-06','2015-11-22','2015-12-08',
  '2015-12-24','2016-01-09','2016-01-25','2016-02-26','2016-03-29','2016-04-14',
  '2016-04-30','2016-05-16','2016-06-17','2016-07-03','2016-07-19','2016-08-04',
  '2016-08-20','2016-09-05','2016-09-21','2016-10-07','2016-10-23','2016-11-08',
  '2016-11-24','2016-12-26','2017-01-11','2017-01-27','2017-02-12','2017-02-28',
  '2017-04-01','2017-04-17','2017-05-03','2017-05-19','2017-06-04','2017-06-20',
  '2017-07-06','2017-07-22','2017-08-07','2017-08-23','2017-09-08','2017-09-24',
  '2017-10-10','2017-10-26','2017-11-11','2017-11-27','2017-12-13','2017-12-29',
  '2018-01-14','2018-01-30','2018-02-15','2018-03-03','2018-03-19','2018-04-04',
  '2018-05-22','2018-06-07','2018-06-23','2018-07-09','2018-07-25','2018-08-10',
  '2018-08-26','2018-09-11','2018-09-27','2018-10-13','2018-10-29','2018-11-14',
  '2018-11-30','2018-12-16','2019-01-17','2019-02-02','2019-02-18','2019-03-06',
  '2019-03-22','2019-04-07','2019-04-23','2019-05-09','2019-05-25','2019-06-10',
  '2019-06-26','2019-07-12','2019-08-13','2019-08-29','2019-09-14','2019-09-30',
  '2019-10-16','2019-11-01','2019-11-17','2019-12-03','1988-03-24','1989-12-08',
  '1991-01-28','1991-02-13','1991-03-17','1991-05-20','1991-12-14','1992-03-19',
  '1993-03-22','1993-12-03','1994-01-20','1994-03-09','1994-09-17','1994-12-22',
  '1995-03-28','1995-09-20','1996-01-26','1997-03-17','1997-04-02','1998-03-04',
  '1998-03-20','1998-06-24','1998-07-26','1999-01-02','1999-03-07','2001-03-28',
  '2001-06-16','2002-01-10','2009-02-14','2009-03-18','2009-05-05','2011-02-04',
  '1988-01-04','1988-01-20','1988-06-12','1988-06-28','1988-08-15','1988-08-31',
  '1988-11-03','1988-11-19','1988-12-05','1989-01-22','1989-02-07','1989-04-12',
  '1989-05-14','1989-05-30','1989-09-30','1989-10-21','1989-12-24','1990-01-25',
  '1990-02-26','1990-03-30','1990-04-15','1990-05-17','1990-06-02','1990-08-21',
  '1990-09-06','1990-10-08','1990-10-24','1991-03-01','1991-04-18','1991-06-05',
  '1991-07-07','1991-08-08','1991-09-09','1991-11-12','1992-02-16','1992-03-03',
  '1992-04-20','1992-06-07','1992-06-23','1992-07-09','1992-10-29','1992-11-14',
  '1992-11-30','1992-12-16','1993-01-17','1993-02-02','1993-02-18','1993-03-06',
  '1993-04-07','1993-05-09','1993-05-25','1993-06-10','1993-07-12','1993-07-28',
  '1993-08-13','1993-09-30','1993-10-16','1993-11-17','1993-12-19','1994-01-04',
  '1994-02-21','1994-04-26','1994-05-12','1994-05-28','1994-06-13','1994-06-29',
  '1994-07-15','1994-07-31','1994-08-16','1994-09-01','1994-10-03','1994-11-04',
  '1994-11-20','1995-01-07','1995-01-23','1995-02-24','1995-03-12','1995-05-15',
  '1995-05-31','1995-06-16','1995-07-02','1995-07-18','1995-10-06','1995-10-22',
  '1995-11-23','1995-12-09','1995-12-25','1996-01-10','1996-02-11','1996-02-27',
  '1996-03-14','1996-03-30','1996-04-15','1996-05-01','1996-06-18','1996-07-04',
  '1996-07-20','1996-08-05','1996-08-21','1996-09-06','1996-09-22','1996-10-08',
  '1996-10-24','1997-01-28','1997-02-13','1997-04-18','1997-05-20','1997-06-05',
  '1997-07-07','1997-08-08','1997-08-24','1997-09-09','1997-10-11','1997-11-28',
  '1998-02-16','1998-04-05','1998-04-21','1998-05-07','1998-05-23','1998-06-08',
  '1998-07-10','1998-08-11','1998-08-27','1998-09-12','1998-09-28','1998-10-14',
  '1998-10-30','1998-11-15','1998-12-01','1998-12-17','1999-01-18','1999-02-03',
  '1999-02-19','1999-03-23','1999-04-08','1999-04-24','1999-05-26','1999-07-29',
  '1999-08-14','1999-08-30','1999-10-17','1999-12-04','1999-12-20','2000-01-05',
  '2000-01-21','2000-02-06','2000-02-22','2000-03-25','2000-04-10','2000-05-12',
  '2000-05-28','2000-06-13','2000-06-29','2000-08-16','2000-09-01','2000-09-17',
  '2000-10-19','2000-11-20','2001-01-07','2001-01-23','2001-02-24','2001-05-15',
  '2001-05-31','2001-07-02','2001-07-18','2001-08-03','2001-09-04','2001-09-20',
  '2001-10-06','2001-10-22','2001-11-07','2001-12-09','2001-12-25','2002-02-11',
  '2002-03-31','2002-05-18','2002-06-19','2002-07-21','2002-08-06','2008-04-16',
  '2008-05-18','2008-06-03','2008-06-19','2008-07-05','2008-10-09','2008-10-25',
  '2008-11-26','2009-03-02','2009-04-03','2009-04-19','2009-05-21','2009-06-06',
  '2009-06-22','2009-07-08','2009-07-24','2009-08-09','2009-08-25','2009-09-10',
  '2009-09-26','2009-10-12','2009-10-28','2009-11-13','2009-11-29','2009-12-15',
  '2010-01-16','2010-03-05','2010-05-24','2010-06-09','2010-06-25','2010-07-11',
  '2010-08-12','2010-08-28','2010-09-13','2010-09-29','2010-10-15','2010-10-31',
  '2010-11-16','2010-12-02','2010-12-18','2011-01-03','2011-01-19','2011-03-08',
  '2011-03-24','2011-04-09','2011-04-25','2011-05-11','2011-05-27','2011-06-12',
  '2011-06-28','2011-07-14','2011-07-30','2011-08-15','2011-09-16','2011-10-02')))

Attach date to rasterstack;

ndvi <- setZ(ndvi, dt[,1], "SampleDate")
dates <- getZ(ndvi)
class(getZ(ndvi)) 
print(ndvi)

> print(ndvi)
class      : RasterStack 
dimensions : 5, 6, 30, 768  (nrow, ncol, ncell, nlayers)
resolution : 30, 30  (x, y)
extent     : 267195, 267375, 4016985, 4017135  (xmin, xmax, ymin, ymax)
crs        : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0 
names      :   layer.1,   layer.2,   layer.3,   layer.4,   layer.5,   layer.6,   layer.7,   layer.8,   layer.9,  layer.10,  layer.11,  layer.12,  layer.13,  layer.14,  layer.15, ... 
min values : -9158.809, -9507.726, -9987.505, -8785.589, -9790.658, -7778.071, -6953.068, -9046.727, -9599.514, -9873.984, -9835.690, -7611.904, -9644.684, -9440.149, -8011.084, ... 
max values :  9885.396,  9260.485,  9699.140,  9089.477,  9684.384,  7493.647,  9712.818,  9222.096,  9387.128,  9642.807,  9988.091,  9719.083,  9426.849,  9824.731,  9932.344, ... 
SampleDate  : 1988-01-04 - 2019-12-03 (range)

The following two functions are required to perform BFAST calculations on this stack.

a <- function(daily.ts) {
  s.month <- round(aggregate(as.zoo(daily.ts), as.yearmon, median), 0)
  s.month <- as.ts(s.month)
  return(s.month)
}

f <- function(data) {
  s <- bfastts(as.vector(data), dates, type = c("irregular"))
  s.d.linear <- round(na.approx(s), 0) 
  s.m.linear <- a(s.d.linear) #above function is used here
  result <- bfast(s.m.linear, h = 0.13, season="dummy", max.iter=2)
  return(c(result$Magnitude, result$Time))
}

Now I use calc() on the ndvi stack:

fbfast <- calc(ndvi, fun = f)

I get the following RasterBrick:

> fbfast
class      : RasterBrick 
dimensions : 5, 6, 30, 2  (nrow, ncol, ncell, nlayers)
resolution : 30, 30  (x, y)
extent     : 267195, 267375, 4016985, 4017135  (xmin, xmax, ymin, ymax)
crs        : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0 
source     : memory
names      :   layer.1,   layer.2 
min values : -3915.188,   156.000 
max values :  6121.308,   196.000 

The results are plotted below, where magnitude and time are shown by layer 1 and layer 2 respectively.

plot(fbfast)

Now I run above process in parallel using Cluster function from Raster package.

beginCluster()
z1 <- clusterR(ndvi, calc, args=list(fun=f), export='a') 

The output is:

> z1
class      : RasterBrick 
dimensions : 5, 6, 30, 2  (nrow, ncol, ncell, nlayers)
resolution : 30, 30  (x, y)
extent     : 267195, 267375, 4016985, 4017135  (xmin, xmax, ymin, ymax)
crs        : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0 
source     : memory
names      :   layer.1,   layer.2 
min values : -7434.631,    34.000 
max values :  4751.379,    59.000 

If I plot(z1) enter image description here

Now I do parallel processing using parallel package;

library(parallel)
c1 <- getCluster()
clusterExport(c1, "a")
z2 <- calc(ndvi, fun=function(x){ t(parApply(c1, x, 1, f))} )

Output is:

> z2
class      : RasterBrick 
dimensions : 5, 6, 30, 2  (nrow, ncol, ncell, nlayers)
resolution : 30, 30  (x, y)
extent     : 267195, 267375, 4016985, 4017135  (xmin, xmax, ymin, ymax)
crs        : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0 
source     : memory
names      :   layer.1,   layer.2 
min values : -7434.631,    34.000 
max values :  4751.379,    59.000 

The output is plotted; enter image description here

The results of z1 and z2 (parallel processing) are same but different from fbfast (without parallel processing). The difference is not in the min and max values only but also the pixels where the changes in magnitude and time happen.

What could be wrong here?

  • Aside from the reported min Max are the results comparable? Maybe inspect them manually in arc/q to see if the values are the same at various pts (or maybe stack them all together and use sampleRandom() in R). I've had r spit out raters with odd min/Max values that seem to be fine otherwise. – Nick Feb 9 at 2:43
  • What other packages are you using? Can you make a reproducible example? – Spacedman Feb 9 at 11:36
  • @Spacedman The code is edited and is reproducible now. – SA Khan Feb 9 at 22:06
1

You need to export the dates variable as well because it is a global

beginCluster()
z1 <- clusterR(ndvi, calc, args=list(fun=f), export=c('a','dates')) 
endCluster()

or

c1 <- getCluster()
clusterExport(c1, c("a","dates"))
z2 <- calc(ndvi, fun=function(x){ t(parApply(c1, x, 1, f))} )
| improve this answer | |
  • Thanks for the correction. I thought dates were attached to NDVI stack and therefore would not be exported to clusters. But you are right and it works fine now. – SA Khan Feb 13 at 21:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.