12

Is there a tool that supports extracting a small region from a bigger GTFS feed? Google's Transitfeed library contains a Merge tool but no Extract/Clip tool, as far as I can see. Other libraries I've found can only read and plot data but don't support manipulation.

Background info: The Transitfeed Feedvalidator shows a lot of errors for my GTFS file. I want to extract my region of interest to assess the local quality.


Recommendations on Twitter pointed me to FME which has both a GTFS reader and a writer. My simple workflow consists of a GTFS reader (bottom left), Shapefile reader (top left, for the area of interest polygon), a clipper and a GTFS writer for the output (only writing the stops and shapes files for now but eventually I'll want all files):

enter image description here

The issue so far is that the output shapes content (orange points) does not contain all the input points (green) which are within the yellow area of interest:

enter image description here

(Data source: https://transitfeeds.com/p/helsinki-regional-transport/735/latest/download)


Using "Create Aggregates" = "No", the clipped shapes seem complete

enter image description here

but the stops definitely are not

enter image description here

  • Is it possible to share the shape file and the workspace you used in the above example? I realize you can't share the GTFS data itself. – Ken Bragg Nov 30 '16 at 17:24
  • The red numbers in the workspace indicate features that could not be clipped. Double clicking on it will open them in the data inspector so you can see what they are. I am guessing they are features with no-geometry. – Ken Bragg Nov 30 '16 at 17:33
  • Yes, those are all the features from the non-geometry files, as far as I can tell. – underdark Nov 30 '16 at 17:35
6

This should be possible in FME 2016. Because we can not write aggregates to GTFS a setting in the Clipper is needed "Create Aggregates" = "No" to prevent FME from creating aggregates when clipped feature results in multiple pieces. The workspace I tested with is here: https://www.dropbox.com/s/b02hd7shefybtmw/gtfs2gtfs.fmw?dl=0 Based on the Helsinki sample data here: https://t.co/wAZHmUS5HN enter image description here

  • Thank you, I'll have to try that back at the office. Btw, is there enough "black magic"/logic to have also the GTFS files that don't contain coordinates by themselves (e.g. routes.txt and trips.txt) clipped? – underdark Nov 30 '16 at 17:34
  • You can join the non-spatial data attributes with a featuremerger or joiner with the official reference developers.google.com/transit/gtfs/reference but not all the Helsinki data is populated so joins will not fully work. – Mapperz Nov 30 '16 at 18:29
  • Yes as @Mapperz points out we need to use the FeatureMerger transformer to join the clipped geometries with non-spatial features. For example trips can be found by using the FeatureMerger to join on shap_id and then calendar records can be found by merging with trip on service_id. I started working through this here: dropbox.com/s/b02hd7shefybtmw/gtfs2gtfs.fmw?dl=0 – Ken Bragg Dec 1 '16 at 9:21
  • 1
    I see the problem and I think the solution ;-) The parent_station attribute on the stops in the Helsinki sample data has some blanks (as allowed in the specification) but they seem to be white space characters so FME will not write them because they don't meet the data type as defined by FME. We are enforcing integer or null. You can fix this using the AttributeManager transformer to set the attribute to null if it is not an integer. See screen shot here: screencast.com/t/dvsWfoYe – Ken Bragg Dec 1 '16 at 12:48
  • 2
    If the above does not help we would be happy to work with you more closely on this via Safe Software support (safe.com/support) and we can post a summary here when we've resolved everything. Or continue here if you prefer. – Ken Bragg Dec 1 '16 at 13:03
5

If you know the routes that are in your region of inteterest, the GTFS Transformer can give you a much smaller GTFS.

In doing so, it will remove all of the trips, shapes, stops, stop_times and calendars that are not included the retained routes.

If you want to remove route_id 1 and keep route_id 3, your transform file would look something like:

{"op":"retain","match":{"file":"routes.txt","route_id":"1"}}
{"op":"remove","match":{"file":"routes.txt","route_id":"3"}}
  • 1
    While not ideal, that could end up being the best available solution. Based on the description, I assume that the Transformer also filters all related files to only contain info of the retained routes. – underdark Nov 30 '16 at 12:02
1

I just came across twalcari/gtfs-filter, a nice utility to filter GTFS by bounding box, time, and/or (classic) travel modes.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.