0

I'm using ogr2ogr to push a large (200,000 feature) dataset to the CartoDB API. I'm using where_clauses to upload 500 features at a time, and sleeping every 20,000 features for a minute to give the server a rest.

Even after implementing this batched behavior, I still occasionally get a 503 error. Are there best practices for pushing data to the API, regarding sleep intervals/number of features to push at once?

I'd like to replicate the shapefile sync capability in cartoDB, but my datasets are too large for shapefiles, and so am using this method instead.

1

Turns out it was an error due to the number of requests I was making to the API. I implemented exponential backoff in my script, and am no longer experiencing the error.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.