0

While trying to export training (about 700 points) and testing data (about 300 points) as TFRecord, I am getting the following error: User memory limit has exceeded

The code I have written, is similar to the material available online, and is:

var band = [ 'B1', 'B2', 'B3','B4','B5','B6','B7', 'B8', 'B9', 'B10', 'B11', 'B12'];
var training = cloudFree.select(band).sampleRegions({
    collection: trainFinal,
    properties: ['CrID'],
    scale: 30,
       tileScale: 16
   });

//Exporting Training and Testing datasets

Export.table.toCloudStorage({
     collection:training,
     description:'Training_Export',
     fileNamePrefix: 'trainingTestData',
     bucket: gcs_bucket,
     fileFormat:'TFRecord'
     });

Is there any mistake in the above code, or should I reduce the number of points / bands?

0

1 Answer 1

1

An out of memory error depends on how much data you're processing. If a lot of data and processing goes into creating cloudFree, you might want to export it as an asset before doing the sampling. That will give you more memory when sampling.

2
  • Hi Daniel, if I need to do this for 15 such images, would you still suggest exporting the images to assets and then carrying out the sampling? Each export to asset takes about 4-5 hours hence making the task laborious Commented May 5, 2020 at 18:00
  • If it fails when you don't export, I suggest you do. You can run your exports in parallel, so while it takes time, it should finish over night. Commented May 5, 2020 at 19:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.