I'm interested to scale CartoDB installation to multiple machines to distribute computational resources. We start using torque with some big tables (more than 5 to 10M points) and the processing for each datacube tile is quite long. Is it possible to have some hints on what you are currently using to scale CartoDB and in particular PostgreSQL/PostGIS?
It depends on how your data changes but you could try to do:
- using a CDN if your data does not change so much
- setup more than one machine (postgres cluster or something like that) and do torque.provider.json fetch tiles from multiple machines (using subdomains for example) or an intermediate proxy with multiple sql-api processes
- try to add indexes to the table by the columns you are using or even precalculate some stuff like (x, y components)