Most probably you have a coastline in your download which is not closed. So the software can not decide on which side of the coastline is water, and on which side is land.
I don't know exatly how TileMill deals with it, but Mapnik renderer uses a separate shapefile with coastlines to avoide that mess.
Maybe you get lucky if you download the data with the JOSM editor, where you can select different areas which will be put together. And you can ask to download border or water relations explicitely complete.
You can also look here http://download.geofabrik.de/europe/france.html for another OSM-based datasource.