The harder initial problem to solve is co-registering your point data with your network. These data may have come from different sources, and so some positional error is to be expected. In the absence of more complex rules governing how points should be located in the network, you can use the closest point on the network to each origin/destination as shown in this existing answer. The challenge here is finding the nearest location along the line, not just the closest vertex:

Once you've done that step, you should have your original network, and the newly co-registered points. From there, you can use the nx_shp function of NetworkX to import the shapefile into a graph model.
And though it isn't documented, if you peek at the source, you'll see that if you create a shapefile with two layers, one of points and one of lines, it will use your points as the nodes, and the lines as the edges, which can be useful.