It sounds like you need this as a generic solution, i.e. having all the world's elevation data available to you for any track you want to process, hence not wanting to store all the CGIAR data locally; the gpsvisualizer.com mentioned above (@underdark@Llaves) may be your best bet.
If you don't need high resolution, the GTOPO data set (1km grid) is only ~300MB for the whole planet; otherwise, the ASTER GDEM (30m) and original SRTM (90m) datasets are available but, as you point out, a lot of data. (The size of the ASTER data can be reduced after download by removing the bundled PDFs which are often larger than the actual elevation data - the Africa dataset was reduced by 40% when I did this!).
In R you can extract the elevation profile from any of these datasets fairly quickly - though loading the raster may take the majority of the time. This uses a small custom readGPX function and gpsbabel to process GPX data:
#Load elevation model and process track:
dem <- raster("E020N40.DEM")
track <- readGPXt("trackfile.gpx")
coordinates(track) <- ~Longitude+Latitude
proj4string(track) <- "+proj=longlat +ellps=WGS84 +towgs84=0,0,0,0,0,0,0 +no_defs"
#Overlay (extract) the elevation data for the track points:
track$profile <- extract(dem, track)
track <- as.data.frame(track)
'track' is now a table of GPS points with lat/lon, other standard GPX data (speed, gps elevation, etc), and a 'profile' column which indicates the elevation at that point.