Since the raster image has fixed cell size which defines the resolution of the image, sometimes it cannot fit within the vector data that you use as a mask. In other words, the image has coarse resolution compared to the vector data.
In order to clip more precised raster, you need to resample the DEM image to a finer resolution. Currently, you are using detailed vector compared to the resolution of the raster image, which create some scattered pixels in areas that cannot fit into detailed vector, especially at the edge of the vector layer.
However, resampling to finer resolution will not give detailed information from the raster DEM, it will just increase the size of the file, so you have to be careful when reducing (resampling) the cell size to finer resolution.