But although I have exhaustively sought what it exactly means I can't figure out to what "long-wavelength" refers to. What is the long and short-wave components of SRTM errors?
In the article from 2006, linked in your question, there is a section named "SRTM Error Sources". It goes into details on each of the three components.
Based on that article, the following answers to your questions can be derived.
Long-wave components of the SRTM errors
An error in knowledge of the baseline roll angle will induce a cross-track slope error in the estimated topography whose magnitude is equal to the roll error. The SRTM instrument used a sophisticated metrology system (AODA) coupled with post-flight filtering and estimation of the baseline position. The main components of the baseline motion are due to the natural modes of oscillation of the mast. These motions can be modeled and removed so that they do not constitute a dominant error source. In addition, the baseline position is affected by the Shuttle’s attitude maneuvers. The time scale for residual roll errors is long, resulting in spatial errors with wavelength on the order of thousands of kilometers. They constitute the primary source for long-wavelength residual errors. Rodriguez et al. (2005) show a representative example of this residual long-wavelength error estimated by subtracting the sea surface topography, which is known to centimetric accuracy from the SRTM estimated topography. The peak values of this residual error are 10 m.
As such, the long-wave component is from residual errors from the roll correction - meaning that it arises from movements of the shuttle.
Short-wave components of the SRTM errors
These errors are due to two sources: random thermal or differential speckle noise and systematic phase changes due to antenna pattern mismatches or long term drift of the instrument electronics. The noise contamination results in height errors which are random and which exhibit short spatial correlation lengths. These errors cannot be compensated during ground processing. The antenna patterns for each of the channels do not have identical far-field phase characteristics. If uncompensated, this phase mismatch results in a net systematic phase error (called the phase screen) which is a function of the look angle. Due to the stability of the antenna far-field phase, this phase screen can be estimated by binning the height errors over the ocean as a function of look angle and applying the mean phase bias as a function of angle as a phase correction to the interferogram. To estimate this phase screen, SRTM collected data prior and after each continental crossing, as well as for a small number of basin-wide ocean data takes. Estimates of the phase screen were obtained as a function of time and for each of the four SRTM elevation beams and all beam positions. The phase screen correction was not observed to change significantly over the mission lifetime; the estimated changes in the height error correction were estimated to be below 10 cm.
Finally, a small slow drift of the differential phase was observed over the lifetime of the mission, probably due to slow changes in instrument temperature. The residual phase errors induce cross-track tilts which are practically indistinguishable from the residual roll errors discussed above.
With the above in mind, the short-wave component comes from two elements types of problems:
(1) short term sensor stuff, aka. thermal noise in the sensor - differential speckle noise - systematic changes due to antenna pattern mismatch
(2) long term sensor stuff, aka. drift of instrument electronics
As for the naming of the error types - the wavelength refers to the scale a which the error types are seen. The "long-wave" components cover thousands of kilometers, while the "short-wave" are small scale random noise.