I want to run BFAST on a raster stack (768 Landsat NDVI) in parallel processing. In order to illustrate the problem, I used the following reproducible code:
Load necessary packages;
library(raster)
library(zoo)
library(xts)
library(bfast)
Create a reproducible rasterstack;
set.seed(123)
r1 <- raster(nrows = 6, ncols = 5, res = 30, xmn = 267195, xmx = 267375, ymn = 4016985, ymx = 4017135)
rr <- lapply(1:768, function(i) setValues(r1, runif(ncell(r1), min = -10000, max = 10000)))
ndvi <- stack(rr)
crs(ndvi) <- "+proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0"
Create date function;
dt <-as.data.frame(as.Date(c(
'2017-01-03','2017-02-04','2018-11-22','2019-05-17','2013-04-14','2013-04-30',
'2013-05-16','2013-06-01','2013-06-17','2013-07-03','2013-07-19','2013-08-04',
'2013-08-20','2013-09-05','2013-10-07','2013-10-23','2013-11-08','2013-11-24',
'2013-12-10','2013-12-26','2014-01-11','2014-01-27','2014-02-12','2014-02-28',
'2014-04-01','2014-04-17','2014-05-03','2014-05-19','2014-06-04','2014-06-20',
'2014-07-06','2014-07-22','2014-08-07','2014-08-23','2014-09-08','2014-09-24',
'2014-10-10','2014-10-26','2014-11-11','2014-11-27','2014-12-13','2014-12-29',
'2015-01-14','2015-01-30','2015-02-15','2015-03-19','2015-04-04','2015-04-20',
'2015-05-06','2015-05-22','2015-06-07','2015-06-23','2015-07-09','2015-07-25',
'2015-08-10','2015-08-26','2015-09-11','2015-09-27','2015-10-13','2015-10-29',
'2015-11-14','2015-11-30','2015-12-16','2016-01-01','2016-01-17','2016-02-02',
'2016-02-18','2016-03-05','2016-03-21','2016-04-06','2016-04-22','2016-05-08',
'2016-05-24','2016-06-25','2016-07-11','2016-07-27','2016-08-12','2016-08-28',
'2016-09-13','2016-09-29','2016-10-15','2016-10-31','2016-11-16','2016-12-02',
'2016-12-18','2017-01-19','2017-02-20','2017-03-08','2017-03-24','2017-04-09',
'2017-04-25','2017-05-11','2017-05-27','2017-06-12','2017-06-28','2017-07-14',
'2017-07-30','2017-08-15','2017-08-31','2017-09-16','2017-10-02','2017-10-18',
'2017-11-03','2017-12-05','2017-12-21','2018-01-06','2018-01-22','2018-02-07',
'2018-02-23','2018-03-11','2018-04-12','2018-04-28','2018-05-14','2018-05-30',
'2018-06-15','2018-07-01','2018-07-17','2018-08-02','2018-08-18','2018-09-03',
'2018-09-19','2018-10-05','2018-10-21','2018-11-06','2018-12-08','2018-12-24',
'2019-01-09','2019-01-25','2019-02-10','2019-02-26','2019-03-14','2019-03-30',
'2019-04-15','2019-05-01','2019-06-02','2019-06-18','2019-07-04','2019-07-20',
'2019-08-05','2019-08-21','2019-09-06','2019-09-22','2019-10-08','2019-10-24',
'2019-11-09','2019-11-25','2001-01-31','2002-03-23','2002-11-02','2007-03-21',
'2008-02-04','2010-05-16','2010-06-17','2012-01-14','2012-12-15','2013-01-16',
'2013-05-08','2013-12-02','2014-03-24','2014-04-25','2014-05-11','2015-02-07',
'2015-02-23','2015-08-02','2016-02-10','2016-03-13','2016-12-10','2018-04-20',
'2018-05-06','2019-01-01','2019-07-28','1999-08-06','1999-09-07','1999-10-25',
'1999-12-12','2000-01-29','2000-02-14','2000-03-01','2000-03-17','2000-04-18',
'2000-05-20','2000-06-05','2000-06-21','2000-08-24','2000-09-09','2000-10-11',
'2000-10-27','2000-11-28','2000-12-30','2001-01-15','2001-03-04','2001-03-20',
'2001-04-21','2001-05-23','2001-06-08','2001-08-11','2001-09-28','2002-01-02',
'2002-01-18','2002-04-08','2002-04-24','2002-06-11','2002-08-30','2002-10-01',
'2003-03-10','2003-03-26','2003-04-27','2003-05-13','2003-05-29','2003-07-16',
'2003-09-02','2003-09-18','2003-10-20','2004-01-24','2004-03-12','2004-03-28',
'2004-04-13','2004-05-15','2004-06-16','2004-07-02','2004-07-18','2004-09-04',
'2004-09-20','2004-10-22','2004-11-07','2004-11-23','2004-12-09','2004-12-25',
'2005-01-10','2005-03-15','2005-03-31','2005-04-16','2005-05-02','2005-05-18',
'2005-06-03','2005-06-19','2005-07-05','2005-07-21','2005-08-06','2005-08-22',
'2005-09-07','2005-09-23','2005-10-09','2005-12-12','2006-04-03','2006-04-19',
'2006-05-05','2006-05-21','2006-06-06','2006-06-22','2006-07-08','2006-07-24',
'2006-08-09','2006-08-25','2006-09-10','2006-09-26','2006-11-13','2007-02-01',
'2007-03-05','2007-04-22','2007-05-08','2007-06-09','2007-06-25','2007-07-11',
'2007-07-27','2007-08-12','2007-08-28','2007-09-13','2007-09-29','2007-10-15',
'2007-10-31','2007-11-16','2007-12-02','2008-01-19','2008-02-20','2008-03-23',
'2008-04-24','2008-05-10','2008-05-26','2008-06-11','2008-06-27','2008-07-13',
'2008-07-29','2008-08-14','2008-08-30','2008-09-15','2008-10-01','2008-10-17',
'2008-11-18','2008-12-04','2009-01-05','2009-01-21','2009-03-10','2009-03-26',
'2009-05-13','2009-05-29','2009-06-14','2009-06-30','2009-07-16','2009-08-01',
'2009-08-17','2009-09-02','2009-09-18','2009-10-04','2009-10-20','2009-11-05',
'2009-11-21','2009-12-07','2009-12-23','2010-01-08','2010-03-13','2010-04-14',
'2010-06-01','2010-07-03','2010-07-19','2010-08-04','2010-08-20','2010-09-05',
'2010-09-21','2010-10-07','2010-10-23','2010-11-08','2010-11-24','2010-12-10',
'2010-12-26','2011-01-11','2011-01-27','2011-03-16','2011-05-03','2011-05-19',
'2011-06-04','2011-06-20','2011-07-06','2011-07-22','2011-08-07','2011-08-23',
'2011-09-08','2011-09-24','2011-10-10','2011-10-26','2011-11-11','2011-11-27',
'2011-12-13','2011-12-29','2012-01-30','2012-02-15','2012-03-02','2012-03-18',
'2012-04-03','2012-04-19','2012-05-05','2012-05-21','2012-06-06','2012-06-22',
'2012-07-08','2012-07-24','2012-08-09','2012-08-25','2012-09-10','2012-09-26',
'2012-10-12','2012-10-28','2012-11-13','2012-12-31','2013-02-01','2013-02-17',
'2013-03-05','2013-03-21','2013-04-22','2013-05-24','2013-06-09','2013-06-25',
'2013-07-11','2013-07-27','2013-08-12','2013-08-28','2013-09-13','2013-09-29',
'2013-10-15','2013-10-31','2013-11-16','2013-12-18','2014-01-03','2014-01-19',
'2014-02-04','2014-02-20','2014-03-08','2014-04-09','2014-05-27','2014-06-12',
'2014-06-28','2014-07-14','2014-07-30','2014-08-31','2014-09-16','2014-10-02',
'2014-10-18','2014-11-03','2014-11-19','2014-12-05','2014-12-21','2015-01-06',
'2015-01-22','2015-03-11','2015-03-27','2015-04-12','2015-04-28','2015-05-14',
'2015-05-30','2015-06-15','2015-07-01','2015-07-17','2015-08-18','2015-09-03',
'2015-09-19','2015-10-05','2015-10-21','2015-11-06','2015-11-22','2015-12-08',
'2015-12-24','2016-01-09','2016-01-25','2016-02-26','2016-03-29','2016-04-14',
'2016-04-30','2016-05-16','2016-06-17','2016-07-03','2016-07-19','2016-08-04',
'2016-08-20','2016-09-05','2016-09-21','2016-10-07','2016-10-23','2016-11-08',
'2016-11-24','2016-12-26','2017-01-11','2017-01-27','2017-02-12','2017-02-28',
'2017-04-01','2017-04-17','2017-05-03','2017-05-19','2017-06-04','2017-06-20',
'2017-07-06','2017-07-22','2017-08-07','2017-08-23','2017-09-08','2017-09-24',
'2017-10-10','2017-10-26','2017-11-11','2017-11-27','2017-12-13','2017-12-29',
'2018-01-14','2018-01-30','2018-02-15','2018-03-03','2018-03-19','2018-04-04',
'2018-05-22','2018-06-07','2018-06-23','2018-07-09','2018-07-25','2018-08-10',
'2018-08-26','2018-09-11','2018-09-27','2018-10-13','2018-10-29','2018-11-14',
'2018-11-30','2018-12-16','2019-01-17','2019-02-02','2019-02-18','2019-03-06',
'2019-03-22','2019-04-07','2019-04-23','2019-05-09','2019-05-25','2019-06-10',
'2019-06-26','2019-07-12','2019-08-13','2019-08-29','2019-09-14','2019-09-30',
'2019-10-16','2019-11-01','2019-11-17','2019-12-03','1988-03-24','1989-12-08',
'1991-01-28','1991-02-13','1991-03-17','1991-05-20','1991-12-14','1992-03-19',
'1993-03-22','1993-12-03','1994-01-20','1994-03-09','1994-09-17','1994-12-22',
'1995-03-28','1995-09-20','1996-01-26','1997-03-17','1997-04-02','1998-03-04',
'1998-03-20','1998-06-24','1998-07-26','1999-01-02','1999-03-07','2001-03-28',
'2001-06-16','2002-01-10','2009-02-14','2009-03-18','2009-05-05','2011-02-04',
'1988-01-04','1988-01-20','1988-06-12','1988-06-28','1988-08-15','1988-08-31',
'1988-11-03','1988-11-19','1988-12-05','1989-01-22','1989-02-07','1989-04-12',
'1989-05-14','1989-05-30','1989-09-30','1989-10-21','1989-12-24','1990-01-25',
'1990-02-26','1990-03-30','1990-04-15','1990-05-17','1990-06-02','1990-08-21',
'1990-09-06','1990-10-08','1990-10-24','1991-03-01','1991-04-18','1991-06-05',
'1991-07-07','1991-08-08','1991-09-09','1991-11-12','1992-02-16','1992-03-03',
'1992-04-20','1992-06-07','1992-06-23','1992-07-09','1992-10-29','1992-11-14',
'1992-11-30','1992-12-16','1993-01-17','1993-02-02','1993-02-18','1993-03-06',
'1993-04-07','1993-05-09','1993-05-25','1993-06-10','1993-07-12','1993-07-28',
'1993-08-13','1993-09-30','1993-10-16','1993-11-17','1993-12-19','1994-01-04',
'1994-02-21','1994-04-26','1994-05-12','1994-05-28','1994-06-13','1994-06-29',
'1994-07-15','1994-07-31','1994-08-16','1994-09-01','1994-10-03','1994-11-04',
'1994-11-20','1995-01-07','1995-01-23','1995-02-24','1995-03-12','1995-05-15',
'1995-05-31','1995-06-16','1995-07-02','1995-07-18','1995-10-06','1995-10-22',
'1995-11-23','1995-12-09','1995-12-25','1996-01-10','1996-02-11','1996-02-27',
'1996-03-14','1996-03-30','1996-04-15','1996-05-01','1996-06-18','1996-07-04',
'1996-07-20','1996-08-05','1996-08-21','1996-09-06','1996-09-22','1996-10-08',
'1996-10-24','1997-01-28','1997-02-13','1997-04-18','1997-05-20','1997-06-05',
'1997-07-07','1997-08-08','1997-08-24','1997-09-09','1997-10-11','1997-11-28',
'1998-02-16','1998-04-05','1998-04-21','1998-05-07','1998-05-23','1998-06-08',
'1998-07-10','1998-08-11','1998-08-27','1998-09-12','1998-09-28','1998-10-14',
'1998-10-30','1998-11-15','1998-12-01','1998-12-17','1999-01-18','1999-02-03',
'1999-02-19','1999-03-23','1999-04-08','1999-04-24','1999-05-26','1999-07-29',
'1999-08-14','1999-08-30','1999-10-17','1999-12-04','1999-12-20','2000-01-05',
'2000-01-21','2000-02-06','2000-02-22','2000-03-25','2000-04-10','2000-05-12',
'2000-05-28','2000-06-13','2000-06-29','2000-08-16','2000-09-01','2000-09-17',
'2000-10-19','2000-11-20','2001-01-07','2001-01-23','2001-02-24','2001-05-15',
'2001-05-31','2001-07-02','2001-07-18','2001-08-03','2001-09-04','2001-09-20',
'2001-10-06','2001-10-22','2001-11-07','2001-12-09','2001-12-25','2002-02-11',
'2002-03-31','2002-05-18','2002-06-19','2002-07-21','2002-08-06','2008-04-16',
'2008-05-18','2008-06-03','2008-06-19','2008-07-05','2008-10-09','2008-10-25',
'2008-11-26','2009-03-02','2009-04-03','2009-04-19','2009-05-21','2009-06-06',
'2009-06-22','2009-07-08','2009-07-24','2009-08-09','2009-08-25','2009-09-10',
'2009-09-26','2009-10-12','2009-10-28','2009-11-13','2009-11-29','2009-12-15',
'2010-01-16','2010-03-05','2010-05-24','2010-06-09','2010-06-25','2010-07-11',
'2010-08-12','2010-08-28','2010-09-13','2010-09-29','2010-10-15','2010-10-31',
'2010-11-16','2010-12-02','2010-12-18','2011-01-03','2011-01-19','2011-03-08',
'2011-03-24','2011-04-09','2011-04-25','2011-05-11','2011-05-27','2011-06-12',
'2011-06-28','2011-07-14','2011-07-30','2011-08-15','2011-09-16','2011-10-02')))
Attach date to rasterstack;
ndvi <- setZ(ndvi, dt[,1], "SampleDate")
dates <- getZ(ndvi)
class(getZ(ndvi))
print(ndvi)
> print(ndvi)
class : RasterStack
dimensions : 5, 6, 30, 768 (nrow, ncol, ncell, nlayers)
resolution : 30, 30 (x, y)
extent : 267195, 267375, 4016985, 4017135 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
names : layer.1, layer.2, layer.3, layer.4, layer.5, layer.6, layer.7, layer.8, layer.9, layer.10, layer.11, layer.12, layer.13, layer.14, layer.15, ...
min values : -9158.809, -9507.726, -9987.505, -8785.589, -9790.658, -7778.071, -6953.068, -9046.727, -9599.514, -9873.984, -9835.690, -7611.904, -9644.684, -9440.149, -8011.084, ...
max values : 9885.396, 9260.485, 9699.140, 9089.477, 9684.384, 7493.647, 9712.818, 9222.096, 9387.128, 9642.807, 9988.091, 9719.083, 9426.849, 9824.731, 9932.344, ...
SampleDate : 1988-01-04 - 2019-12-03 (range)
The following two functions are required to perform BFAST calculations on this stack.
a <- function(daily.ts) {
s.month <- round(aggregate(as.zoo(daily.ts), as.yearmon, median), 0)
s.month <- as.ts(s.month)
return(s.month)
}
f <- function(data) {
s <- bfastts(as.vector(data), dates, type = c("irregular"))
s.d.linear <- round(na.approx(s), 0)
s.m.linear <- a(s.d.linear) #above function is used here
result <- bfast(s.m.linear, h = 0.13, season="dummy", max.iter=2)
return(c(result$Magnitude, result$Time))
}
Now I use calc() on the ndvi stack:
fbfast <- calc(ndvi, fun = f)
I get the following RasterBrick:
> fbfast
class : RasterBrick
dimensions : 5, 6, 30, 2 (nrow, ncol, ncell, nlayers)
resolution : 30, 30 (x, y)
extent : 267195, 267375, 4016985, 4017135 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
source : memory
names : layer.1, layer.2
min values : -3915.188, 156.000
max values : 6121.308, 196.000
The results are plotted below, where magnitude and time are shown by layer 1 and layer 2 respectively.
Now I run above process in parallel using Cluster function from Raster package.
beginCluster()
z1 <- clusterR(ndvi, calc, args=list(fun=f), export='a')
The output is:
> z1
class : RasterBrick
dimensions : 5, 6, 30, 2 (nrow, ncol, ncell, nlayers)
resolution : 30, 30 (x, y)
extent : 267195, 267375, 4016985, 4017135 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
source : memory
names : layer.1, layer.2
min values : -7434.631, 34.000
max values : 4751.379, 59.000
If I plot(z1)
Now I do parallel processing using parallel package;
library(parallel)
c1 <- getCluster()
clusterExport(c1, "a")
z2 <- calc(ndvi, fun=function(x){ t(parApply(c1, x, 1, f))} )
Output is:
> z2
class : RasterBrick
dimensions : 5, 6, 30, 2 (nrow, ncol, ncell, nlayers)
resolution : 30, 30 (x, y)
extent : 267195, 267375, 4016985, 4017135 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=43 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
source : memory
names : layer.1, layer.2
min values : -7434.631, 34.000
max values : 4751.379, 59.000
The output is plotted;
The results of z1 and z2 (parallel processing) are same but different from fbfast (without parallel processing). The difference is not in the min and max values only but also the pixels where the changes in magnitude and time happen.
What could be wrong here?
