You don't really need polygons, and in fact points for your origins and destinations would work better. You can have a point represent a building or parking lot, but it sounds like you're working at a pedestrian scale in which case it might be better to have points for specific building entrances or other such destinations.
The most important thing with your lines to make them useable as a network involves snapping. Be sure all lines are connected and snapped to each other. Furthermore, make sure that where two lines meet, if that is supposed to be an intersection, both have a shared vertex there. If they are not supposed to meet (for example a second floor connecting walkway crosses between buildings over a path outside/underneath) they should not have a shared vertex. Ideally, lines should be broken into separate lines at any intersection (four way intersection, four lines not two crossing).
From there it depends on your analysis and definition of 'optimal'. The lines will by default have the property of length, so you can do shortest path analysis. For time, you need some attribute of travel time for that edge, which you could create by dividing the length by a speed factor. Beyond that you're looking at other network edge attributes - maybe it's steep and has a cost penalty for that, or maybe you have a rating system for pleasantness to walk that factors into how favorable a route is.
I would suggest running through exercises 1-3 and 12 of the Network Analyst tutorial to get a feel for how things work.