Achieving this goal is somewhat a basic task in GIS, however the method in QGIS might not be trivial. Your best chance is to use GRASS's
r.walk function, which creates an anisotropic cost surface (dem+slope+other factors).
First, you have to create a friction surface as an input to
r.walk. In your case it can be a single-valued raster (1.0) matching the extent of your DEM. You can create it with
r.mapcalculator with the formula:
A*0+1 where A is your DEM.
Next, you have to select a set of starting points from your CSV. These are the points, the accumulated cost surface will be calculated from. You have to create an individual cost surface from every starting point. It might be smart to define the ending points associated with every starting point in this step (in individual layers off course). After, you can run
r.walk with the created inputs. The starting points can be in a single layer, you can iterate through them with the green arrow in the dialogue box.
Now in an ideal case, you have the cost surfaces and the ending points for every cost surface. In theory you could find the least cost paths with
r.drain, but for me, it ended up in an error (python couldn't import the QgisRaster library). If you run in the same issue, you can go with SAGA's "Least cost paths" algorithm. It will create a point and a line layer for every ending point with the cost surface (use the iteration button again). After you have all the lines, you can merge them into a single shapefile with SAGA's "Merge shapes layers" tool.
This method can be very slow with the increment of points, so if you have a lot of them, you might try to automatize the method with python. It will be still a lot of time to calculate (especially the cost surfaces), but you don't have to create tons of ending point layers manually.