I tried "Image Boundary" tool in QGIS but it fails with some rasters.

Another option that worked for me is:-

1. Convert input raster to a single digital number by using gdal_translate command-line utility.

gdal_translate -co compress=lzw -b 1 -ot byte -scale 1 1 inp.tif out.tif

This will create raster with values of 1 everywhere inp.tif had non-zero values.

2. Then in ArcMap you can use "raster to polygon" tool to get polygon(shapefile).

Output shapefile will have non-quadrilateral boundaries, with multiple polygon vertices, following image's actual boundary.

I tried "Image Boundary" tool in QGIS but it fails with some rasters.

Another option that worked for me is:

1. Convert input raster to a single digital number by using gdal_translate command-line utility.

gdal_translate -co compress=lzw -b 1 -ot byte -scale 1 1 inp.tif out.tif

This will create raster with values of 1 everywhere inp.tif had non-zero values.

2. Then in ArcMap you can use "raster to polygon" tool to get a polygon  (shapefile).

Output shapefile will have non-quadrilateral boundaries, with multiple polygon vertices, following image's actual boundary.