CartoDB import api: upload data from stdin instead of file

Question

I am trying to upload data using the CartoDB import api. Uploading a regular file works but my attempts to upload data from stdin are failing.

The example of uploading a file works perfect.

curl -F file=@/Users/mbj/Downloads/cvr_count.csv "https://{CARTODBUSER}.cartodb.com/api/v1/imports/?api_key={APIKEY}"

I want to load data comming from stdin instead.

According to curl man page

To read content from stdin instead of a file, use - as the filename. This goes for >both @ and < constructs.

To test this:

cat /Users/mbj/Downloads/cvr_count.csv | curl -F file=@- "https://{CARTODBUSER}.cartodb.com/api/v1/imports/?api_key={APIKEY}"

Response:

{"item_queue_id":"7b8477f9-0926-4f11-b898-c9d9f149ae00","success":true}

But the table never appears in CartoDB. Has anybody tried this? Am I doing something wrong ?

Answer 1

After you launch the import job you need to check the state of that import using

curl "https://{CARTODBUSER}.cartodb.com/api/v1/imports/7b8477f9-0926-4f11-b898-c9d9f149ae00?api_key={APIKEY}"

in this case it will return something like:

{
    "content_guessing": false,
    "data_type": "file",
    "error_code": 1002,
    "get_error_text": {
        "title": "Unsupported/Unrecognized file type",
        "what_about": "Should we support this filetype? Let us know in our <a href='mailto:support-suggestions@cartodb.com'>support email</a>!"
    },

What is happening under the hood is ogr2ogr, the tool we use to import files can't recognize the file type.

For the moment CartoDB is not able to guess file types