1

I'm trying to write raster bricks using lapply but I get the following error:

Error in .local(.Object, ...) : Dataset copy failed
Calls: lapply ... <Anonymous> -> new -> initialize -> initialize -> .local
Execution halted

Here is my entire code:

    library(raster)
    library(dismo)
    library(parallel)
    
    setwd("/scratch/bntjoa002/chelsa_cmip5_ts/")
    
    # set the number of cores
    #detectCores()
    mc.cores=80
    
    #get model names
    #modelnames<-c("ACCESS1-3", "MIROC5", "CESM1-BGC", "CMCC-CM") #could also read a csv file of the model names
    modelnames<-c("ACCESS1-3")
    
    #experiment names
    experiments<-c("rcp45", "rcp85")
    
    #specify years
    start.year<-2070
    end.year<-2089
    
    # start the loop to calculate bioclim vars
    
    for (i in 1:length(modelnames)){
      
      model = modelnames[i]
      
      for(j in 1:length(experiments)){
        
        experiment<-experiments[j]
        
        tasmax<-brick(paste0(getwd(),'/tasmax','/CHELSAcmip5ts_tasmax_',model,"_",experiment,"_",start.year,"-",end.year, ".nc"),
                      varname='air_temperature')
        tasmin<-brick(paste0(getwd(),'/tasmin','/CHELSAcmip5ts_tasmin_',model,"_",experiment,"_",start.year,"-",end.year, ".nc"),
                      varname='air_temperature')
        pr<-brick(paste0(getwd(),'/pr','/CHELSAcmip5ts_pr_',model,"_",experiment,"_",start.year,"-",end.year, "_V1.1.nc"),
                  varname='precipitation_flux')
        
        
        # make a vector with the years in climate data time series
        years<-seq(start.year,end.year,1)
        
        nyears=length(years)
        
        # make index for the year
        ind=sort(rep(1:nyears,12))
        
        # Calculate bioclim variables for each year using mclapply
        bcbricks=mclapply(unique(ind),function(x){
          #extract just the climate data for each year from the raster brick
          tasmax.year=tasmax[[which(ind==x)]]
          tasmin.year=tasmin[[which(ind==x)]]
          precip.year=pr[[which(ind==x)]]
          # calculate bioclim variables
          biovars(precip.year,tasmin.year,tasmax.year)
        },mc.cores=mc.cores)
        
        
        #if you want to save all bioclim vars for a year as a stack for each year
        #For FUTURE climate projections, stack all bioclim variables for each year
        # if(experiment=="rcp45"| experiment=="rcp85"){
        
        # output directory
        #   outDir=paste0('/ClimateModelData_NASA_GDDP_Bioclim/future/',model)
        
        #   lapply(seq_along(years),function(x){
        #     writeRaster(bcbricks[[x]],file=paste0(getwd(),outDir,'/',"Bioclim1-19","_",model,"_",experiment,"_",years[x],'.tif'))
        #   })
        # }
        
        #if you want to save one bioclim var as a stack across all years
        if(experiment=="rcp45"| experiment=="rcp85"){
          
          # output directory
          outDir=paste0('/scratch/bntjoa002/chelsa_cmip5_ts/',model)
          
          # stack individual bioclim variables across years
          lapply(1:nlayers(bcbricks[[1]]),function(x){
            
            bcvar.by.allyears=stack(lapply(bcbricks,function(y){ y[[x]]}))
            
            writeRaster(bcvar.by.allyears,file=paste0(getwd(),outDir,'/',names(bcbricks[[1]])[x],'_',model,'_',experiment,'_',start.year,'_',end.year,'.tif'))
          })
        }
        print(j)
      }
      print(i)
    } 

I'm submitting the job to a cluster, and there is definitely enough space on the drive. Any ideas what the issue might be?

1
  • 2
    We can't run this code so you'll have to dig into it to work out what's happening more specifically. I'd add a bunch of message calls to see how far the code is getting. If you suspect the error is from inside your lapply near the end, print out the path you are trying to write the file to before the writeRaster. Check that the folder it is trying to write to exists. Its easy to get an extra or missing slash when constructing paths with paste functions - use file.path instead. Print out summary info about bcvar.by.allyears before writing to check its what you expect.
    – Spacedman
    Nov 13, 2020 at 9:02

2 Answers 2

1

The issue was indeed that the path to the folder was incorrect. For writeRaster, I had getwd() as well as "outDir", the latter already having the path to the directory in it (so the path was repeated). I removed "getwd()" from writeRaster, and now I no longer get the "dataset copy failed" error.

1

I also encountered this problem when using "raster" and "smapr" to process data. Error in .local(.Object, ...) : Dataset copy failed. For me, it was because I cleaned ~/.cache manually.

I solved this problem by rebuilding the path in the following way:

$ cd  ~/.cache/
$ mkdir smap

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.