Looks like this is a Mercator problem. Although your bounding box looks like a square, the lengths of the sides are actually unequal. In the spherical Mercator projection, which is cylindrical, scale is maintained longitudinally, but increases latitudinally. For example, at 30 degrees latitude, a degree of latitude is nearly 69 miles, while a degree of longitude is only like 60 miles. So the height of your square is actually less than the width. When your square is rotated 90deg, the former width is now the height, and it has to be stretched to maintain distance. And the former height is now the width, and it has to be contracted to maintain distance. Rotating again to 180deg stretches and contracts in reverse, returning to the original shape.
If you want the dimensions of the rotated box to be predictable, I suggest working in an equidistant projection, like 3786, and then switching back to 3785 later.