0

I've got what I believe to be a properly configured CartoDB instance set up on a CentOS7 virtual server. I've managed to get all the components compiled and running, including redis-server, the two node app.js files and the two http server processes all in a development configuration. I can login and access the Carto instance without issue or error. However, once I attempt to import a new dataset (have tried both csv and geojson using data here: https://zenodo.org/record/165520) I get the following error from the resque instance:

2017-03-16 20:53:49 UTC: ogr2ogr call:            OSM_USE_CUSTOM_INDEXING=NO PG_USE_COPY=YES PGCLIENTENCODING=UTF-8   -f PostgreSQL  PG:"host=localhost port=5432 user=development_cartodb_user_781299ac-9773-4629-a298-625bb85af550 dbname=cartodb_dev_user_781299ac-9773-4629-a298-625bb85af550_db password=[snip]development_cartodb_user_781299ac-9773-4629-a298-625bb85af550" -t_srs EPSG:4326  -lco DIM=2 -lco PRECISION=NO /tmp/imports/20170316-10095-19txyzg/city_farms_gardens_1.0.geojson  -nln cdb_importer.importer_a7a32a120a8a11e79805005056a34934 -nlt PROMOTE_TO_MULTI  -doo PRELUDE_STATEMENTS="SET statement_timeout TO '1h'" -doo CLOSING_STATEMENTS='SET statement_timeout TO DEFAULT' -update
2017-03-16 20:53:49 UTC: ogr2ogr output:          sh: -f: command not found

2017-03-16 20:53:49 UTC: ogr2ogr exit code:       32512
2017-03-16 20:53:49 UTC: Ogr2ogr FAILED!
2017-03-16 20:53:49 UTC: ogr2ogr.exit_code = 32512
2017-03-16 20:53:49 UTC: ogr2ogr.command = OSM_USE_CUSTOM_INDEXING=NO PG_USE_COPY=YES PGCLIENTENCODING=UTF-8   -f PostgreSQL  PG:"host=localhost port=5432 user=development_cartodb_user_781299ac-9773-4629-a298-625bb85af550 dbname=cartodb_dev_user_781299ac-9773-4629-a298-625bb85af550_db password=[snip]development_cartodb_user_781299ac-9773-4629-a298-625bb85af550" -t_srs EPSG:4326  -lco DIM=2 -lco PRECISION=NO /tmp/imports/20170316-10095-19txyzg/city_farms_gardens_1.0.geojson  -nln cdb_importer.importer_a7a32a120a8a11e79805005056a34934 -nlt PROMOTE_TO_MULTI  -doo PRELUDE_STATEMENTS="SET statement_timeout TO '1h'" -doo CLOSING_STATEMENTS='SET statement_timeout TO DEFAULT' -update
2017-03-16 20:53:49 UTC: ogr2ogr.command_output = sh: -f: command not found

2017-03-16 20:53:49 UTC: Errored importing data from /tmp/imports/20170316-10095-19txyzg/city_farms_gardens_1.0.geojson:
2017-03-16 20:53:49 UTC: CartoDB::Importer2::LoadError: Ogr2ogr ERROR

Now to my eyes "ogr2ogr.command_output = sh: -f: command not found" would seem to indicate that it can't find ogr2ogr, but I've confirmed that this is in /usr/local/bin and this in turn is in $PATH. Have tried with both ogr2ogr 1.3 and 2.1 with the same result.

cartodb/config/app_config.yml is only lightly customised, so the line

defaults: &defaults
  # If uncommented, most images like static map previews and twitter card image urls will use this CDN urls
  #cdn_url:
  #  http:             "http.cdn.host"
  #  https:            "https.cdn.host"
  http_client_logs: true
  ogr2ogr:
    binary:           'which ogr2ogr2.1'
    csv_guessing:     true
  debug_assets: true
  mandatory_keys:     [layer_opts, sql_api, varnish_management, redis, session_domain]
  session_domain:     'carto-dev.mapping.community'
  # If activated, urls will use usernames in format //SESSION_DOMAIN/user/USERNAME and ignore subdomains if present
  subdomainless_urls: true
  http_port:           3000 # nil|integer. HTTP port to use when building urls. Leave empty to use default (80)
  https_port:          # nil|integer. HTTPS port to use when building urls. Leave empty to use default (443)
  secret_token:       '[snip]'
  account_host:       'localhost.lan:3000'
  account_path:       '/account'
#  vizjson_cache_domains: ['.localhost.lan']

The command which ogr2ogr2.1 returns "/usr/local/bin/ogr2ogr21" (because I've copied it there) and manually running the command

ogr2ogr -f PostgreSQL PG:"host=localhost port=5432 user=development_cartodb_user_781299ac-9773-4629-a298-625bb85af550 dbname=cartodb_dev_user_781299ac-9773-4629-a298-625bb85af550_db password=[snip]development_cartodb_user_781299ac-9773-4629-a298-625bb85af550" -t_srs EPSG:4326 -lco DIM=2 -lco PRECISION=NO ./city_farms_gardens_1.0.geojson -nln cdb_importer.importer_a7a32a120a8a11e79805005056a34934 -nlt PROMOTE_TO_MULTI -doo PRELUDE_STATEMENTS="SET statement_timeout TO '1h'" -doo CLOSING_STATEMENTS='SET statement_timeout TO DEFAULT' -update

provides no error (note above has been modified to work on locally downloaded version of geojson file I'm trying to import rather than from /tmp).

What am I missing here? Happy to provide further information, log/config snips etc. if that would be useful.

Improve this question

1 Answer 1

Reset to default
1

First of all, I think that CARTO user group is better for this kind of question: [email protected] . It'd be great if you sent it there too :_)

That error usually happens because of a wrong ogr2ogr binary configuration, as you pointed. Nevertheless, yours looks ok, so I'd suggest you checking the following:

  • which ogr2ogr2.1 with the user that runs Resque. Maybe it's at your $PATH but not at the one that runs it.
  • Have you restarted Resque since the last configuration change? Maybe you restarted the server but not the queues.
Improve this answer
1
  • Thanks @juanignaciosl. I've cross-posted as suggested and will post back here with the answer once I've managed to resolve the issue. I've also added a few details there which is to note that I'm testing with the same user account that is executing Resque and restarting the service after each configuration change. So still stuck on both counts.
    – Jeremy Kidwell
    Commented Mar 18, 2017 at 15:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.