I need to find potential routes for undersea power cables in South East Asia, therefore I need to select all areas where the ocean is less than 4000m deep. I load the "ocean basemap" from ESRI but it looks like it is only a picture and no attribute table with data about how deep the ocean is at an specific point. Is there a way to create a attribute table out of a map or do I have the wrong map?
As Stefan says, you have the wrong data, the basemap is just an image with no elevation data.
A quick and easy way would be to download the 0m and 4000m Bathymetry data sets from Natural Earth, and clip the 0m shapefile with the 4000m shapefile (use Erase tool in ArcGIS or Difference in QGIS).
Or if you need more detailed raster data, there are a few options such as SRTM30_PLUS v8 from eAtlas, or SRTM15+ V2.1 from the OpenTopography website. There may be more detailed bathymetry available in your area of interest depending on coverage and local government policy on open data.
There are basemaps that show elevation above MSL with contours for mountains and the elevation numbers (meters) are part of the basemap which show up when you zoom-in. For example opentopomap.org has them. Obviously not interactive where you can see elevation where ever you want, but static numbers in many areas on the contours themselves.