0

I am running a spatial panel regression in R using spml from package splm. When I conduct this on about 1/200th of my data, the program executes. But when I try to do so on a third of my data, R uses up all the memory on the terminal I am working on and the program crashes. My code for specifying the spatial regression is below. How would I make this function operate efficiently enough to produce results and not crash? My terminal has 22 core, so parallel processing is likely an option - I just don't know how to perform this with spml.

results = spml(fm, 
                data=dtaSub,
                listw=weightsList,
                model="within", 
                effect = "twoway",
               spatial.error = "b")
```
Improve this question
2
  • How big is your data? Many spatial methods have to compute a full NxN distance matrix and so have quadratic memory costs, and you'll be off the scale pretty quickly. Have you looked at alternative methods designed for big data? Can you take random samples and pool estimates from splm?
    – Spacedman
    Jan 20, 2021 at 8:45
  • My data consists of about 400,000 tiles (resolution: 0.01 degrees) over 12 years. I've been working with about 1/3 of this data (130,000 tiles, 12 years) and the shapefile is 185 MB. The data itself spans a large land area: each tile is in a 30 km buffer around a point of interest, but these points of interest appear across a subcontinent. I am familiar with some of the big data methods for regression, but I am here to learn if there are similar methods that can be used with splm.
    – C. Ashley
    Jan 20, 2021 at 15:34

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy