I have a csv file of latitudes and longitudes. Is it somehow possible that all these lat-long values can be marked on Google Earth?
Google Earth reads KML files, so you need to create a KML file from your CSV.
There are online CSV-KML converters. I don't know how good they are.
Or you could load it into QGIS and save it as KML.
By default you'll probably get standard Google Earth markers for each point, but with a bit more effort you can style them and label them as you wish.