Batch processing 300 FBX files with textures using FME?

Question

My goal is to provide a contractor with FBX files with textures with a file size that is roughly 10MB.

I have esri ASC files for my Area Of Interest(AOI) as well as a recent aerial capture. My AOI fits within 300 500m square tiles.

I had thought that the best way of doing this would be to use the RasterTiler and RasterPropertyExtractor transformers to give me files named with the bottom left of the tile so that the file name could be used to merge the imagery with the surface. Here is my file name expression e@Value(_lower_left_x)n@Value(_lower_left_y)

So I now have 300 files of each ASC, JPG and WLD files for the JPGs.

I can create the desired output using a TINGenerator and AppearanceSetter followed by a FBX writer.

I had to manually enter a matching JPG file to the JPG reader and fme_basename value seems to get lost passing through the TINGenerator. So it writes files with blank names ".fbx" and ".fbm" dir for texture.

I feel I should be able to use the FilenamePartExtractor transformer to pass _rootname to the second reader, but I haven't been able to make this happen. It might be easier to point the reader at the ECW mosaic of the entire AOI and pass it the geom of the footprint of the ASC tile to be clipped

I have found this article on batch processing which I intend to follow once I can get the above process working.

Answer 1

I think the easiest thing you can do to retain fme_basename (or fme_dataset would work too) is to use it as a group-by attribute in the TINGenerator. I'm guessing that transformer drops the fme_basename attribute because if multiple features get merged into a single surface, then it would no longer be applicable. So as long as you are OK with each source feature producing a separate surface (which I believe is exactly what you are trying to do here), you can use the group-by option.

Then you can replace the JPEG reader with a FeatureReader. Connect the TINGenerator:TINSurface port to the FeatureReader and use fme_basename/fme_dataset to define the JPEG file to read (you could use FilenamePartExtractor on fme_dataset to construct the actual file name first).

From the FeatureReader the JPEG data goes into the AppearanceSetter:Appearance port, while the port goes into the Geometry port.

But, as you mentioned, the other solution would be to use the batch process method and a WorkspaceRunner. One workspace reads a list of files and passes that to the above workspace to deal with, one at a time. Then there's no concern about group-by.

The best thing about that solution is that processing is more efficient because you aren't trying to handle all the data at once. Additionally, the WorkspaceRunner lets you have multiple processes in parallel, speeding it up even more.

Answer 2

This is the workflow and I came up. I suspect VariableSetter and VariableRetriever are superfluous as Mark doesn't mention them in his answer, however in my workbench they were necessary.

Using FeatureReader with

\\mymachine\jpg_tiles\@Value(fme_basename).jpg

as the dataset of the FeatureReader meant it read the JPG with the same name as the ASC file. In conjunction with the workspace runner transformer in a separate workbench I was able to loop over a directory of files, select the matching JPG, create the TIN, set the texture and write to a FBX file.