What is that borders in the water on OpenStreetMap maps?
Is this some of maritime zones?
Is this "territorial waters", i.e. 12 nautical miles away from borders?
What is between islands?
Where/how to obtain this information?
The boundary=maritime OpenStreetMap Wiki page might help. Basically, solid lines show the territorial waters as you guessed (12 nautical miles except between states).
A good way to check the nature of an object (line, point,...) in OpenStreetMap is to open the map editor (I think you have to register yourself) and select the object. Lots of information will be displayed in the left panel: