For the labels use Overpass Turbo to query the Openstreetmap data for what information you want. If you are new to this service or want a different way to do the same thing use the Quick OSM plugin to do the same (it in turn uses Overpass).
If you have some familiarity with OSM then this should be a relatively simple process - at least if you are looking for point data for cities, towns, suburbs, etc. You'd be looking for places I think: (place = country ; place = region ; place = city ; place = town ; etc). The data might need an edit... after all the location for such a point is going to be relatively arbitrary, and exactly what names fit what categories is often going to be a personal choice related to local circumstances.
Style the points to have a label but no marker. I've never done this but remember reading that it could be done - I think presumably by making a very small marker and/or covering it with the label.
Don't forget the credit for this data if the mapping is for distribution.
If you also need to generate a simple country/region outline you can probably get this using the same route.
If you want is something more than this - perhaps a good map showing lots of detail but in grey and lacking many normal labels then you might need to look at more specific solutions or combinations of solutions. There's a pretty good chance that someone out there already has a style of Openstreetmap based mapping which does something like what you want. Check out link1; link2 and link3 to scratch the surface. Whether this is available for free, for a fee, or whether you need to get them to adapt it a little will require some research.