I cannot replicate this error so, I imagine, as the error indicates, you are actually running out of memory. Besides reading in the grid, with 229,374 polygons you are trying to create 68,812,200 sample points. A few things to check are how much RAM you have and if you are running the 64-bit version of R (within RStudio). I would note that a computer with even a relatively small amount of RAM should be able to hand this problem leading me to think that you are running 32-bit R.

Here is the code that I used and it is running fine with R 4.1.0 x86_64-w64-mingw32/x64, sf_1.0-2, sp_1.4-5 and spatialEco_1.3-7.

library(spatialEco)
library(sp)
library(sf)

shp <- as(sf::st_read("C:/test/grid.shp"), "Spatial")
random_poly = sample.poly(shp, n = 300, type = "random")

You can proof the code by reducing the size of your problem.

( random_poly = sample.poly(shp[sample(1:nrow(shp), 10),], 
                            n = 10, type = "random") ) 

This opens the door to subsampling your problem down. In a loop, you can grab a few thousand polygons at a time, create a sample and write them out. The sf::st_write function has an append argument that will allow you to add to iteratively append a shapefile on disk. Something along these lines should work, will take awhile but control memory usage.

( n=round(nrow(shp) /20, 0) )
g <- split(1:nrow(shp), ceiling(seq_along(1:nrow(shp)) / n))

st_write(as(sample.poly(shp[g[[1]],], n = 300, type = "random"),
         "sf"), "sample_pts.shp")  

lapply(g[-1], function(x) {  
  st_write(as(sample.poly(shp[x,], n = 300, 
           type = "random"), "sf"),
           "sample_pts.shp",
           append=TRUE) })

I cannot replicate this error so, I imagine, as the error indicates, you are actually running out of memory. Besides reading in the grid, with 229,374 polygons you are trying to create 68,812,200 sample points. A few things to check are how much RAM you have and if you are running the 64-bit version of R (within RStudio). I would note that a computer with even a relatively small amount of RAM (4GB) should be able to hand this problem leading me to think that you are running 32-bit R or having RAM allocated elsewhere (another process).

Here is the code that I used and it is running fine with R 4.1.0 x86_64-w64-mingw32/x64, sf_1.0-2, sp_1.4-5 and spatialEco_1.3-7.

library(spatialEco)
library(sp)
library(sf)

shp <- as(sf::st_read("C:/test/grid.shp"), "Spatial")
random_poly = sample.poly(shp, n = 300, type = "random")

You can proof the code by reducing the size of your problem.

( random_poly = sample.poly(shp[sample(1:nrow(shp), 10),], 
                            n = 10, type = "random") ) 

This opens the door to subsampling your problem down. In a loop, you can grab a few thousand polygons at a time, create a sample and write them out. The sf::st_write function has an append argument that will allow you to add to iteratively append a shapefile on disk. Something along these lines should work, will take awhile but control memory usage.

( n=round(nrow(shp) /20, 0) )
g <- split(1:nrow(shp), ceiling(seq_along(1:nrow(shp)) / n))

st_write(as(sample.poly(shp[g[[1]],], n = 300, type = "random"),
         "sf"), "sample_pts.shp")  

lapply(g[-1], function(x) {  
  st_write(as(sample.poly(shp[x,], n = 300, 
           type = "random"), "sf"),
           "sample_pts.shp",
           append=TRUE) })