4

I need a little help using the WorkspaceRunner in FME.

Essentially I want to batch run polygon shapefiles in one folder (30 of them) against another load of files, in the form of a tiled grid (26 tiles).

Unfortunately you can't just put two merged feature type readers into 1 workspace and press Run, the results are all wrong due to the way that FME is reading in the files and processing them. I need to send one shapefile at a time to process against all 26 tiles. Enter the WorkspaceRunner!

This is the main processing workspace, 'Workspace 1';

Workspace 1

It's not a complicated workflow, just bringing in some Grid tiles and some polygons and doing some joining, taking some attributes and writing out to a .dbf. The writer is fanned out by a unique combo of tile name and Year attribute from the polygon shapefile.

This is the WorkspaceRunner, 'Workspace2';

workspace 2

Using the Directory and File Pathnames reader (PATH), i am reading the individual file names in the folder where the 30 shapefiles for analysis are kept. In the WorkspaceRunner itself, the .fmw for Workspace 1 is identified, the location of the 26 tiles is auto-inputted into the top Source parameter and the parameter for the location to be written to is also auto-inputted (which should contain 30 shapefiles against 26 grids = 780 .dbf files).

Now, the parameter field for the 2nd lot of shapefiles, i.e. the 30 polygon shapefiles, is causing problems. The results will not go beyond the 1st shapefile, i am only getting 26 results. I have used path names, the $(SourceDataset_PATH) user parameter, the path_filename attribute and i get a misture of log error messages or incorrect output (ie only 26 files). The regular log error message is confusing; i am told no .shp exist in the nominated folders, when they clearly do, but when using the User Parameter as shown in the picture, that error goes away. However, being shapefiles, there are pathnames for .prj, .dbx files etc etc and it's just not working!

does anyone have any advice at all!?! thanks

(FME 2014, 32-bit, Build 14230)

(I've read the FMEpedia and FME blog posts about this to no avail)

  • 1
    windows? i use path_windows variable from filereader to start workspace using it as SourceDataset_DGN8 (one file at time) (in you case that is probably SourceDataset_SHP ). Define those grid files in Workspace so its simpler to run. – simplexio May 21 '14 at 13:20
  • thanks (im using windows); using path_windows creates an error " SHAPE Reader: Dataset `C:\xxxxxxxxxxx\shapefile1.dbf' lacks the required extension shp " but im pretty sure this is because i had set the Path Filer parameter to * instead of *.shp . Im running a test now and will reinform soon. – Sam May 21 '14 at 13:25
  • @ simplexio, see answer now – Sam May 21 '14 at 13:43
1

Finally solved it;

the WorkspaceRunner must have the pathname attribute in the Source Esri Shape File(s) parameter box; i.e. the 'path_windows' attribute (not 'path_filename' or any other)

BUT, in conjunction with this, when using the PATH reader, you must set the path filter parameter (available when setting up the PATH reader to begin with) to *.shp and not just *. This ensures that the files pushed forward to the other workspace dont include the .prj and .dbf etc which will just error.

The reader in the workspace you are calling in the WorkspaceRunner - set to be informed by the master workspace in the paragraph above - needs to be a Merged Feature Type with a merge filter of *.

enter image description here

  • another option for filtering is use tester/testfilter to extension. good when you need join data based on filename – simplexio May 22 '14 at 6:33
  • yes, that's a good suggestion and actually you can then filter them away to different WorkspaceRunners. My next task is to read out all files in given folders, listed within a text file.... – Sam May 22 '14 at 7:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.