I'm having issues with using iterate on a featurecollection.

I have reason to believe that one of my methods downstream is duplicating elements, so I want to assign sequential integers to my rows to keep track of them and to ensure that there are no duplicates being added.

However, the only changes that I'm making is this integer row. When I do this and try to export this FeatureCollection, the API returns the error that the object is too large.

I'm using the iterate function to assign the sequential IDs. To test, I mapped a function that added a random number to the ID column, and the asset had no problem being exported. So I think something is wrong with the iterate function below:

nhd = nhd.sort('AreSqKm', false);
var first = ee.List([ee.Feature(null, {"N": -1, "Remove": true})]);

//Add these IDs to each image. 
var addID = function(feat, list){
  var num = ee.Number(ee.Feature(ee.List(list).get(-1)).get("N"));
  feat = ee.Feature(feat).set({"N": num.add(1)});

//Iterate through and add ID, Merge to images. 
var tl = ee.FeatureCollection(ee.List(nhd.iterate(addID, first)));
nhd = tl.filterMetadata("Remove", "not_equals", true);

  collection: nhd, 
  description: "NHD_addID", 
  assetId: "NHD_addID"

Any time I try this, I get the error "Error: Encoded object too large", and a google search does not help too much in finding a detailed explanation. Am I messing something up with the .iterate function?


The problem is that your addID iteration function, in order to produce its output, is building up an ee.List of every feature in the collection. This will always fail if the collection is too large to fit in memory at once.

Unfortunately, I don't know of a way to attach an iteratively computed property to a collection in the way you want. The closest feature that exists is .randomColumn() but as the name suggests, that adds random numbers, not sequential ones.

However, there might be another method: every feature in a collection has a unique .id() string (also known as the property system:index). It's not guaranteed to be sequential — the format you get depends on the table you started with and also the processing steps — but each of those steps that might merge/join/flatten/etc. two collections (and therefore be able to duplicate features) will systematically produce guaranteed unique IDs for each one, by adding text rather than by renumbering.

For example, suppose that your original table had feature IDs like 0000000000000000325b (actual example from WDPA). Then if you did some kind of computation that matched a feature two different ways and put both of them in the result, you'd find that they have IDs like 1_0000000000000000325b and 2_0000000000000000325b or similar. Thus, you can look for the pattern of your original IDs and see if there are any duplicates of those.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.