I cannot replicate this error so, I imagine, as the error indicates, you are actually running out of memory. Besides reading in the grid, with 229,374 polygons you are trying to create 68,812,200 sample points. A few things to check are how much RAM you have and if you are running the 64-bit version of R (within RStudio). I would note that a computer with even a relatively small amount of RAM (4GB) should be able to hand this problem leading me to think that you are running 32-bit R or having RAM allocated elsewhere (another process).
Here is the code that I used and it is running fine with R 4.1.0 x86_64-w64-mingw32/x64, sf_1.0-2, sp_1.4-5 and spatialEco_1.3-7.
library(spatialEco)
library(sp)
library(sf)
shp <- as(sf::st_read("C:/test/grid.shp"), "Spatial")
random_poly = sample.poly(shp, n = 300, type = "random")
You can proof the code by reducing the size of your problem.
( random_poly = sample.poly(shp[sample(1:nrow(shp), 10),],
n = 10, type = "random") )
This opens the door to subsampling your problem down. In a loop, you can grab a few thousand polygons at a time, create a sample and write them out. The sf::st_write function has an append argument that will allow you to add to iteratively append a shapefile on disk. Something along these lines should work, will take awhile but control memory usage.
( n=round(nrow(shp) /20, 0) )
g <- split(1:nrow(shp), ceiling(seq_along(1:nrow(shp)) / n))
st_write(as(sample.poly(shp[g[[1]],], n = 300, type = "random"),
"sf"), "sample_pts.shp")
lapply(g[-1], function(x) {
st_write(as(sample.poly(shp[x,], n = 300,
type = "random"), "sf"),
"sample_pts.shp",
append=TRUE) })