I've done some work on a prototype that extracts data from the UK Open Survey public DEM data sets for Britain. They have their DSM and DTM files downloadable in ASC format which is quite easy to read into Unity via a C# StreamReader loop. The data can each be stored as float[,] which can easily be applied to Unity terrain heightmaps.
Also using these two arrays of floats, I created a diff array which can then be filtered by a "tolerance" of say 5 metres to figure out likely roof lines, tall trees, etc. If you then run an algorithm to find the edges, you can extrude downwards from the DSM height to the DTM floor to get some approximate building footprints and a point cloud of roof heights. From those you can make some basic Unity meshes for the building outlines. At some point I intend to go back and write an algorithm to calculate where the straight planes are in order to reduce vertices inside a plane.
Open Survey UK also provide a bunch of other data sets, such as Open Map Local, which contains road information and records of building footprints that I found to be somewhat more complete than Open Street Map. I downloaded this data in GML format, which again is quite amenable to being read in by StreamReader (or XmlReader if you want to do the work with XML attributes etc). Other formats such as ESRI and LAZ files, I found harder to get hold of decent file format information, and there are a number of libraries you could probably pull into Unity to handle many of those. I grabbed DotSpatial.Projections from nuget, for example, and that just goes right into a Unity Plugins/ folder and works off the bat. (For nuget libraries with dependencies you may need to put in more effort to get those working.) Anyway with the GML data for roads and buildings, I wrote a parser that basically converted the "geometry" tags into vectors and went from there.
As for other sources of DEM that might not be in ASC or similarly useful formats, one trick is to download and install QGIS, and save out layers in the format you prefer. I'm not an expert in GIS or that software, but it seemed to work when I tried it for an ESRI shapefile. I'd recommend you can also use the command line lastools utilities to pre-convert other types of files such as LAZ and turn them into ASC, rather than try to process the LAZ format natively in Unity itself.
TLDR advice I suppose would be: obtain (or pre-convert) DEM or shape geometry data in a text format that you can read into Unity and then process it further from there.
One other cool thing I tried in the prototype was hooking up the resulting terrain with a Google Street View (all you need is to sign up to a free Google Maps developer API key) and using DotSpatial to convert the British National Grid coordinates to WGS84 to get the street view panorama. I made a render camera for the street view tiles and presto - looking around the blobby textures and seeing how they roughly lined up with the street view actual image!
Anyway, hope that helps and gives you some idea about approaches to try.