DSM void filling results into problems with pixels (using ArcMap)

Question

Before writing here I exhausted any online resources, blogs, videos etc I could find. Here's my problem, with photos and a thorough explanation.

I have a very high res DSM (5m) that has holes (voids). I need to keep and use this surface and I am searching for a way to make it adequate for the usage I want it, that is for hiking maps. Searching for a replacement of lower res is not a solution for my case. Also, I am currently using ArcMap and do not spare the time to switch to ArcGIS Pro right now.

Trying to fix this, I came up with the following workaround that still produces some visible inadequacies (see pictures 4 and 5):

1. I smoothed the DSM 5m with focal statistics in order to smooth the edges of the voids
2. I used a DTM 30m (ALOS) which I clipped at the same extent and then resampled at 10m.
3. I combined my DSM 5m and the DTM 10m with map algebra (raster calculator) with the following expression: Con(IsNull("DSM_5m"),"DTM_10m","DSM_5m")
4. I examined the result and even smoothed it with focal statistics (circle, 3px radius), but it is still inadequate.

My problem is: at the edges of the void fill, the merging/stitching is very visible in surfaces like hillshade or slope and derivatives like contour lines, a result which is not acceptable for hiking maps. In the photos below I pinpointed a few pixel areas where the problem is very visible.

My question is: is there a way to fix this, i.e. smooth the edges before or after combining the two rasters in order to fill the voids?

Perhaps I have to start over with another procedure, in case I am doing something wrong or not really helpful (like combining the two rasters)

The rasters I'm combining. The DTM 30m (only visible here in the holes as grayscale) fills the voids of the DSM 5m

The rasters I'm combining. Hillshade for DSM 5m

The rasters I'm combining. Hillshade for both rasters.

Showing the problem at the edges of the void fills

Showing the problem at the edges of the void fills #2

Answer 1

The idea is simple: sacrifice parts of both models to create smooth transitional one. This post is duplicate of that, but solution below favors high resolution model.

Create outlines of holes buffers, make it equal to relevant cell sizes, i.e. 5 and minus 30 m. Note example below uses (1 and -20 m):

Interpolate lines using relevant DEMs, and triangulate surface:

I used feature to polygon and selection by attributes to define polygons between green and red lines. They were used as a mask when I extracted transition model from TIN and merged it with high resolution one:

Fill what remains with low resolution, although I suggest experimenting with TIN, or topo to raster using same 2 buffers mentioned above and cells inside green polygon as points.