In the ArcGIS Desktop Help on Saving to previous versions of ArcGIS it says:
In most cases, after you open and save an existing map document (.mxd
file), the map can no longer be opened with earlier versions of ArcGIS
because it will now reflect the new functionality. Similarly, new
documents cannot be opened in earlier versions of the software;
however, you can use the Save A Copy command to make a copy of a
map document so you can open and work with it in previous versions of
ArcGIS. Note that documents from ArcGIS 10.1, 10.2, and related
releases, such as 10.2.1, are directly compatible with each other;
ArcGIS 9.0 and 9.1 documents are also compatible with each other.
Conquently, I think you need to open your 10.5 MXD at 10.5 or later and save it to a 10.4 MXD. I am not aware of any earlier ArcGIS version or non-ArcGIS way to do it.
There is an existing ArcGIS Idea titled Backward/Forward Compatibility which you may wish to vote for, if you have not already.