First you need a map of world polygons.
> m = getMap()
Then you need your points in a spatial format. I'll make some random points on the globe and create a SpatialPointsDataFrame using the
sp package and making sure they have the same projection string as the map:
> set.seed(123); d = data.frame(long=runif(100,-180,180), lat=runif(100,-90,90))
> coordinates(d) = ~long+lat
> proj4string(d) = proj4string(m)
Now I can extract any of the columns from the map data corresponding to each point with
 Jamaica <NA> <NA> <NA> <NA>
 <NA> <NA> <NA> Angola <NA>
 <NA> <NA> Antarctica <NA> <NA>
<NA> are points in the ocean.
I can add the
NAME to the source points:
> d$country = over(d,m)$NAME
1 (-76.47209, 17.99801) Jamaica
2 (103.7898, -30.09176) <NA>
3 (-32.76831, -2.049654) <NA>
4 (137.8863, 81.80529) <NA>
5 (158.5682, -3.077569) <NA>
6 (-163.5997, 70.26304) <NA>
The oceans are big.
Note that: exact world boundaries would be ridiculously complex detailed coastline and border lines. This data set is a simplification. If your points are near the coast they might get mapped to the oceans, or if near a border they might get mapped to the wrong country. Do not use for military planning purposes. Also this is a snapshot of the world at a particular time, countries change boundaries and sovereignties.
If you don't have zillions of these to do and can afford to wait for a web service to give you the answers, use
geonames - you will have to get a free geonames user id from geonames.org and then loop over your data:
> GNcountryCode(17.99801, -76.47209)