I thoroughly edited the answer to give a more complete picture. The edited answer also reflects what I discussed in the comments.
There is not only a single answer to your question.
First of all, I have to say that it would indeed be possible to use the observed range measurements (which are encoded in the form of range bins - mostly referring to the column dimension in SAR image matrices) and the perfectly known sensor positions to calculate some information about the topography. However, the range measurements are limited by the system's range resolution, which depends on the (range) bandwidth of the sensor. And that bandwidth is inherently technically limited. Even for the most sophisticated airborne SAR systems, the (slant range) resolution does not get a lot better than 1-2 decimeters. For spaceborne systems such as Sentinel-1, the range resolution (e.g. in standard IW mode) is only 20 meters. And that certainly is not sufficient to any useful terrain reconstruction task.
That being said, you can read up on something called SAR imaging geodesy. This technique was developed in the last years and allows single-image-based positioning of scatterers with cm accuracy - without any phase exploitation! In exchange, many deterministic corrections, e.g. regarding ionospheric and tropospheric delay, solid earth tides etc. need to be calculated and applied.
Good references are, for example,
Eineder, M., Minet, C., Steigenberger, P., Cong, X., & Fritz, T. (2011). Imaging Geodesy--Toward Centimeter-Level Ranging Accuracy With TerraSAR-X. IEEE Transactions on Geoscience and Remote Sensing, 49(2), 661-671.
Cong, X., Balss, U., Eineder, M., & Fritz, T. (2012). Imaging geodesy—Centimeter-level ranging accuracy with TerraSAR-X: An update. IEEE Geoscience and Remote Sensing Letters, 9(5), 948-952.
Now back to the necessity of phase analyisis or interferometry: By exploiting the phase via SAR interferometry, you can reconstruct relative distance differences to a fraction of the wavelength, which can bring you down to the cm or even mm domain. To stay again wit the Sentinel-1 example, the system's C-band wavelength is about 5.5 cm, so the theoretically achievable accuracy is much better than what could be achieved with the range measurements alone - albeit at the cost of relativity: SAR interferometry does not provide absolute measurements, and in order to reconstruct topography, you need at least one reference point that defines the height datum.
Now, why can't you just use the phase of a single image, why do we need interferometry at all? Unfortunately, the observed phase does not ONLY relate to the distance traveled by the signal, but also to a contribution that comes from the backscattering properties of the target's material. Since the observed phase is the sum of distance-related phase and scattering phase, it appears to be random in a single SAR image.
If you calculate the phase difference of two coherent SAR images, the scattering phase nicely cancels out, and thus you end up with only the distance-related phase difference (most often called topographic phase difference):
(observed phase 1) = (distance related phase 1) + (scattering related phase)
(observed phase 2) = (distance related phase 2) + (scattering related phase)
(interferometric phase) = (observed phase 1) - (observed phase 2) = (distance related phase 1) + (scattering related phase) - (distance related phase 2) - (scattering related phase) = (distance related phase 1 ) - (distance related phase 2)
Of course, the assumption that the scattering related phase doesn't change must hold - but this is most usually the case if the two SAR images are precisely coregistered (which they must be anyway to ensure coherence).
In this context, it needs to be repeated that in a SAR image the only thing that is recorded is the amplitude A and the phase phi of the observed signal. While the phase would directly relate to the distance between antenna and target, the problem is that phase values are inherently ambiguous, as they are only defined in the interval [-pi;+pi]. Thus, you always need to "unwrap" your phase values before you can relate them back to distance and thus - knowing your antenna position precisely - the observed topography.
Furthermore, all I have said relates to rather simple terrain with slopes that are not to steep. As soon as you face SAR-inherent geometric effects such as layover that appear for strongly sloped terrain elements (steep mountains, buildings), you'll need more sophisticated techniques based on the interferometry principle (such as SAR tomography) to separate the observations mixed in a layover pixel by analyzing the phase.
Last but not leas, it has to be mentioned that "the beauty of SAR" is not in its capability to map terrain, i.e. 3D information, but in monitoring changes of the topography over time (i.e. 4D information) by the interferometric phase analysis. This allows to measure deformations with mm accuracy from distances of more than 500 km away - no other measurement technique is able to do so!