This is a very limited solution. As long as your route is in a straight line you can line up your coverage layer polygons with the exact same overlap each time. Use the ID or a page number column to get the order number and set up an atlas. Don't hide the coverage layer.
On your second page where you will have two overlaps, one for the first page and one for the second. add a rectangle matching up with the overlap of the next and previous polygon. This is why it's limited, as long as the overlap is always the same, these rectangles don't have to move so they can remain static. The only issue will be with the first and last page where there will be only one overlap.
Now add a text box with the expression
@atlas_pagename +1 and
@atlas_pagename -1 to the next and previous overlay boxes respectively. This will give the next page number. You might need to alter your expression to use
@atlas_featurenumber or whatever, your just trying to get the page numbering for the next and previous page.
Effectively you should have a page with two overlaps that now number up and down as you move through the atlas. But what about the first and last page?
Select either the text box or rectangle first (you're going to have to do this step for them anyway). Go to the 'Position and size' properties of the text box (i'll use this one for the instructions) and take note of the 'X' property. Under data defined override add the expression:
CASE WHEN @atlas_pagename = '[you last page number]' THEN [a large number] ELSE [the current value of X] END
This will move the text box left or right off the page when it is the last page and move it back onto the page when it is any other page number. Now repeat for the other text box and rectangles. Again change the expression to reflect however you have your page numbering set up.
This should hopefully reduce any more work needed to create the atlas. But with a little extra you could do the same with a more complex route. Use the data defined override of the 'X' and 'Y' properties of the text boxes and rectangles and move them around the page depending on the page number.