FME Limit features

Question

I have a FME in which i am using http caller to call an API. This API is returning me more then 10,000 features. FME is running fine. Now i want to do some transformation on few attributes coming from API. The problem is everytime i run it i have to wait for 10,000 features to complete, which takes a long time. I want some way to just limit it to first 10 records only so that i can test what i am doing with transformers instead of running it for whole data.

I tried using Counter and then tester to just pass feature count less then 10 but it still processes the remaining features.

Answer 1

Don't know if this is exactly what you want but you can use the Sampler with "First N" to just send only those further down for processing. I'm not sure if this gets to the heart of your issue though because all 10,000 will still be processed up until the point of the sampler.

If you're looking to do iterative development, I'd strongly suggest trying out the "Feature Caching" that is part of FME 2018 and later. Check out https://www.safe.com/blog/2018/05/caching-data-fme-evangelist174/ for an introduction, or our training material for more info. (This knowledge base article is also worth a read.) With this, you'd run with caching on and all the processing would be done on all the features up until some point. Then you can just "run from there" as you iterate and develop. I'd still use the Sampler in this workflow and just adjust things downstream from the sampler, using partial runs until you get the results you like, then take out the Sampler.