2

I'm running some process for about 48 hours and I'm not sure about what is going on or maybe better - how should I interpret the log.

Please have a look at attached part of the log. Just after successful ? clipping (at least is something I can read from log ?) the FME executes "performing low-level intersection" and repeat that part at least about 200 times in my case. Is it correct ? Should I patiently wait to finish that part of the process or it somehow stuck in that place ?

Thanks in advance for help !

Finished clipping 303150 / 525807 clippees against all clippers
Performing low-level intersection at phase #1... 100% done
Setting up lines for phase# 2 ... 100% done
Setting up lines for phase# 2 ... 100% done
Performing low-level intersection at phase #1... 100% done
Setting up lines for phase# 2 ... 100% done
Setting up lines for phase# 2 ... 100% done
Performing low-level intersection at phase #1... 100% done
... Last line repeated 2 times ...
Setting up lines for phase# 2 ... 0.48% done
Setting up lines for phase# 2 ... 100% done
Setting up lines for phase# 2 ... 100% done
Performing low-level intersection at phase #1... 100% done
... Last line repeated 16 times ...
Setting up lines for phase# 2 ... 0.48% done
Setting up lines for phase# 2 ... 100% done
Setting up lines for phase# 2 ... 100% done
Performing low-level intersection at phase #1... 100% done
Setting up lines for phase# 2 ... 0.48% done
Setting up lines for phase# 2 ... 100% done
Setting up lines for phase# 2 ... 100% done

Thanks for remarks. Two data sources: Geotiffs (about 13k of tiles) and water areas features (about 600k) in shapefile. The bounding boxes of geotiff's in the first part of the workflow should clip the WA features. According to info from Dale process all DEM tiles at once raster feature's don't hold any data... so I believe I'm not processing DEM data at all and in the process only them bounding boxes are used. Then it should be simply use of clipper with clippers and clippees both being vector features. This is also a place of the process that I believe (interpreting the log) the process stuck so I'm not describing the further part of the workbench. If any won't find the extra information enough I'm able to share with the workbench which will be more self-explanatory for the rest of the process (can't find any button to attach it here).

Improve this question
4
  • You should try and add some more detail to your question, i.e. What are you trying to run through (MapInfo TAB Files, ESRI Shape etc etc), what are you doing with them (translating to a new format/coord sys etc.), how big the table you are trying to process is. The more info the better someone will be able to help!
    – Rob Quincey
    Commented Aug 1, 2012 at 7:48
  • 2
    Screenshot of the Workbench workspace would be good - annotated of course..
    – Mapperz ♦
    Commented Aug 1, 2012 at 13:19
  • Clipper can be very expensive, especially without parallel processing. A screen shot of the workplace would be helpful to figure out if there is another way to do it.
    – blord-castillo
    Commented Aug 1, 2012 at 18:28
  • 1
    How about running with a smaller dataset until you're sure of the workbench? You could clip all input to a tiny bounding box and then just see how the fully contained features are processed. It sounds like this is a new workbench so probably you'll need to make a few changes before running it on the full dataset
    – tomfumb
    Commented Aug 31, 2012 at 21:21

1 Answer 1

Reset to default
1

If it's still running then there's no reason to think it won't continue to completion. However, you'll want to check system resources. If you've run out of memory then there might be some disk caching going on, which would really slow you down. If that's the case the tip below might help....

It would be good to see the workspace, but one performance tip with Clipper is to use the Clippers First option. There is some more information on that here and here.

The second of those links is talking about the NeighborFinder transformer, but the concept is still the same. Essentially, by forcing all the clip boundaries to be first into the transformer, the clippees don't need to be stored/cached in memory. It can be a vast performance improvement.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.