You have a lot of segments, no specific order or orientation. You do not know which actually touch or overlap, and which are merely close.

For each segment only the begin and endpoint is important. The goal is to make one big polyline, the orientation of that polyline is not important, i guess.

In that case i would make some kind of set/array of segments, start with the first one, which is entirely random. 

In pseudocode do something like

    all_segments = set of all segments

    # take the first segment out of set
    new_polyline = all_segments.pop
    
    until all_segments.empty?
      start_segm = find_segment_closest_to(new_polyline.start_point)
      remove_from_all_segments(start_segm)
      expand_polyline_at_begin(new_polyline, start_segm) 
      end_segm = find_segment_closest_to(new_polyline.end_point)
      expand_polyline_at_end(new_polyline, end_segm)
      remove_from_all_segments(end_segm)
    end

Something like that?
That is a very high level. You will need to handle boundary cases.
I am guessing you know or could the biggest possible gap/distance, because you will need to be able to somehow exclude points being found: if the closest possible point is at the other end of the polyline than it is not an option :) The easiest way to handle that is to define a maximum gap-distance. This will also limit the number of points you will have to look at for eacht segment.

Does that help? I hope it gets you started.