Why do I get a 503 error when pushing data to the CartoDB SQL API

Question

I'm using ogr2ogr to push a large (200,000 feature) dataset to the CartoDB API. I'm using where_clauses to upload 500 features at a time, and sleeping every 20,000 features for a minute to give the server a rest.

Even after implementing this batched behavior, I still occasionally get a 503 error. Are there best practices for pushing data to the API, regarding sleep intervals/number of features to push at once?

I'd like to replicate the shapefile sync capability in cartoDB, but my datasets are too large for shapefiles, and so am using this method instead.

Answer 1

Turns out it was an error due to the number of requests I was making to the API. I implemented exponential backoff in my script, and am no longer experiencing the error.