Why is the curl syntax unable to create datastore?

Question

I am implementing the following code to upload an existing shapefile as per the documentation,

curl -u admi PUT 'Content-type: application/zip' \
      --data-binary @/var/www/geo/shapefile/zip/Poa_horridula.zip \
      http://localhost:8080/geoserver/rest/workspaces/acme/datastores/Poa_horridula/Poa_horridula.zip

But I am getting following error,

curl: (6) Couldn't resolve host 'Content-type'

No such datastore: acme,Poa_horridula

The document says that the curl code will create the datastore then why am I getting this error?

UPDATE: I changed the code to upload a zip file but I am still not able to upload any shapefile.

Where am I going wrong?

UPDATE 2: Ok I am almost there

So I changed the command to,

curl -u admin:geoserver -v -XPUT -H 'Content-type: application/zip' --data-binary @/var/www/geo/shapefile/zip/Poa_horridula.zip http://localhost:8080/geoserver/rest/workspaces/acme/datastores/Poa/file.shp

The only problem is that I am able to add the shapefile but the layer is not being published. Is that normal? Do I have to manually publish it?

Answer 1

I think the file URL is the culprit.

'file:./shapefile/csv_shp/Stemmodontia_reticulata.shp'

I'm not sure Geoserver supports that URL. Where is the file actually located?

---

No such datastore: opengeo,Stemmodontia_reticulata

Check if the opengeo workspace exists in your server. This error could be due to that.

If you're interested in creating a datastore from a directory of existing shapefiles, you can try something like the following

curl -u admin:geoserver -XPUT -H 'Content-type: text/plain' \
   -d 'file:///data/shapefiles/' \
   "http://localhost:8080/geoserver/rest/workspaces/acme/datastores/shapefiles/external.shp?configure=all" 

That way you won't have to add the files one by one.

You'll have to add the workspace first.

curl -u admin:geoserver -v -XPOST -H 'Content-Type: application/xml' \
     -d '<coverageStore><name>int_dec</name><workspace>Stemmodontia_reticulata</workspace>  
         <enabled>true</enabled></coverageStore>' \
         http://localhost:8080/geoserver/rest/workspaces/Stemmodontia_reticulata/coveragestores

Then add the existing shapefile

curl -u admin:geoserver -XPUT -H 'Content-type: application/zip' \
   --data-binary @Stemmodontia_reticulata.zip \
   http://localhost:8080/geoserver/rest/workspaces/opengeo/datastores/Stemmodontia_reticulata/file.shp

I hope that works.

Answer 2

This is the final solution. So I was using shapefile.py to create the shapefile from CSV files and the problem is that shapefile.py does not create a .prj file and that was the reason that the shapefile was not publishing. Here is the code for creating the .prj file for EPSG:4326 (you just need to define the projection, epsg in the code below, you need and it will create the desired .prj file for you),

prj = open("/path/shapefile/%s.prj" % filename, "w")
    epsg = 'GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563]],PRIMEM["Greenwich",0],UNIT["degree",0.0174532925199433]]'
    prj.write(epsg)
    prj.close()

And then you can run the below cURL commands to publish the shapefiles as needed.

For Uploading an Existing Shapefile, use this code,

curl -u admin:geoserver -v -XPUT -H 'Content-type: text/plain' -d 'file:/var/www/geo/shapefile/csvQshp/Quercus_iltisii.shp' http://localhost:8080/geoserver/rest/workspaces/acme/datastores/Quercus_iltisii/external.shp?

Give the complete path of your file and do not forget ? after the external.shp.

For Uploading a Zipped Shapefile, use this code,

curl -u admin:geoserver -v -XPUT -H 'Content-type: application/zip' --data-binary @/var/www/geo/shapefile/zip/Poa_horridula.zip http://localhost:8080/geoserver/rest/workspaces/acme/datastores/Poa/file.shp

Do not forget @ before the file path for zipped files.

For Uploading a complete Dir of Shapefiles, use this code,

curl -u admin:geoserver -v -XPUT -H 'Content-type: text/plain' -d 'file:/var/www/geo/shapefile/csvQshp/' "http://localhost:8080/geoserver/rest/workspaces/acme/datastores/csvQshp/external.shp?configure=all"

Answer 3

Since you already have the uploading section, let me add on to that.

When debugging cURL, I usually use something that echoes things back since for the life of me I cannot remember the cURL syntax correctly. httpbin helps me a lot

For example, to debug uploading a shapefile which is done by:

 curl -u admin:geoserver -v -XPUT -H 'Content-type: text/plain' -d 'file:/var/www/geo/shapefile/csvQshp/Quercus_iltisii.shp' http://localhost:8080/geoserver/rest/workspaces/acme/datastores/Quercus_iltisii/external.shp?

I would modify send it to httpbin like this:

 (test)rburhum@peru.local ~ $ curl -u admin:geoserver -v -XPUT -H 'Content-type: text/plain' -d 'file:/var/www/geo/shapefile/csvQshp/Quercus_iltisii.shp' http://httpbin.com/put/geoserver/rest/workspaces/acme/datastores/Quercus_iltisii/external.shp
* About to connect() to httpbin.com port 80 (#0)
*   Trying 67.215.65.132...
* connected
* Connected to httpbin.com (67.215.65.132) port 80 (#0)
* Server auth using Basic with user 'admin'
> PUT /put/geoserver/rest/workspaces/acme/datastores/Quercus_iltisii/external.shp HTTP/1.1
> Authorization: Basic YWRtaW46Z2Vvc2VydmVy
> User-Agent: curl/7.24.0 (x86_64-apple-darwin12.0) libcurl/7.24.0 OpenSSL/0.9.8r zlib/1.2.5
> Host: httpbin.com
> Accept: */*
> Content-type: text/plain
> Content-Length: 55
> 
* upload completely sent off: 55 out of 55 bytes
* HTTP 1.0, assume close after body
< HTTP/1.0 303 See Other
< Location: http://httpbin.com/put/geoserver/rest/workspaces/acme/datastores/Quercus_iltisii/external.shp
< Content-Length: 0
< Connection: close
< Date: Wed, 14 Nov 2012 20:11:56 GMT
< Server: OpenDNS Guide
< 
* Closing connection #0

This tells me that Basic Authorization happened and that an upload happened with a PUT verb to the Location specified there. Helpful

Now to answer your publishing question, I usually use this curl python method:

def publish_layer(store_name, table_name, style_name):
    payload =  "<featureType><name>{0}</name></featureType>".format(table_name)

    c = pycurl.Curl()
    c.setopt(c.URL, '{0}/rest/workspaces/{1}/datastores/{2}/featuretypes'.format(GEOSERVER_URL, GEOSERVER_WORKSPACE, store_name))

    c.setopt(pycurl.USERPWD, GEOSERVER_USER + ':' + GEOSERVER_PASSWORD)
    c.setopt(pycurl.POST, 1)
    c.setopt(pycurl.HTTPHEADER, ["Content-type: text/xml"])
    c.setopt(pycurl.POSTFIELDSIZE, len(payload))
    c.setopt(pycurl.READFUNCTION, StringReader(payload).read_cb)
    b = StringIO.StringIO()
    c.setopt(pycurl.WRITEFUNCTION, b.write)

    c.perform()

    check_http_code(c, (table_name, style_name),b)

    return