So after some research I decided to stick with the OSM metadata. I found the Overpass API, which is basically a query language, which I managed to understand reading this:
and testing it here
I ended up with this query:
This finds all the roads I considered public using a highway filter within my bounding box. Then takes those ways and loads their nodes - (._;>;); - such an obvious statement, isn't it!
This gives me the necessary metadata to use for filtering. Now I can construct geometries and intersect with my tracks using .Net Topology Suite