Consider trying the QGIS > Raster > Conversion > Polygonize (Raster to Vector) tool. Where adjacent raster cells contain the same value (in your example, the value 5), this tool dissolves out their shared boundary, resulting in fewer output polygons. Taken to its logical extreme, if all of the raster cells contain the same value, the vector output will contain only the outer boundary.
Sidenote discussion: In my experience with both QGIS and Arc desktop, converting rasters to vectors will often result in output that has geometry errors. I've never bothered to find out what those errors are because QGIS has the very speedy and effective tool Fix Geometries. This is a "black-box" operation in that you simply supply the vector layer's name and the tool does the magic wash. Thus, I always run Fix Geometries after a raster-to-vector conversion. If I knew how to code, I suppose that I could open up the tool's hood and see what's going on underneath. Arc desktop has a similar tool, Repair Geometry, but I've had situations (admittedly few) where it failed to completely fix things, which were subsequently solved by "Fix Geometries". I'm at QGIS 3.18.2 and Arc 10.8.1.