0

I'm looking to automate the creation of GeoServer (2.18) Stores and Layer publishing and hitting an issue.

Using:

curl -v -u user:password -XPUT -H "Content-type: application/zip" -d @many-shp-layers.zip "http://192.168.1.10/geoserver/rest/workspaces/mytestworkspace/datastores/shapefiles/external.shp?configure=all"

This produces the following:

* TCP_NODELAY set
* Connected to 192.168.1.10 port 80 (#0)
* Server auth using Basic with user 'user'
> PUT /geoserver/rest/workspaces/mytestworkspace/datastores/shapefiles/external.shp?configure=all HTTP/1.1
> Host: 192.168.1.10
> Authorization: Basic bGl2Omxpdg==
> User-Agent: curl/7.61.1
> Accept: */*
> Content-type: application/zip
> Content-Length: 336609
> Expect: 100-continue
>
< HTTP/1.1 100
* We are completely uploaded and fine
< HTTP/1.1 500
< X-Frame-Options: SAMEORIGIN
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Date: Tue, 23 Feb 2021 03:38:29 GMT
< Connection: close
<
* Closing connection 0
Error while storing uploaded file:

My understanding from the documentation is that it should accept a zip file containing more than one layer? If that's incorrect, what would the curl command look like to handle a directory (on Unix) of shapefiles instead? I've tried many iterations but the above is the closest I've got.

I thought the external.shp?configure=all might be the issue. I tried removing that but returned a HTTP 405 (Method Not Allowed).

[Edit]

Here is the log output from Geoserver:

    2021-02-23 23:13:44,339 ERROR [geoserver.rest] - Error while storing uploaded file:
org.geoserver.rest.RestException 500 INTERNAL_SERVER_ERROR: Error while storing uploaded file:
    at org.geoserver.rest.catalog.AbstractStoreUploadController.handleFileUpload(AbstractStoreUploadController.java:90)
    at org.geoserver.rest.catalog.DataStoreFileController.doFileUpload(DataStoreFileController.java:577)
    at org.geoserver.rest.catalog.DataStoreFileController.dataStorePut(DataStoreFileController.java:263)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:190)
    at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:138)
    at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:105)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:893)
    at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:798)
    at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
    at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1040)
    at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:943)
    at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
    at org.springframework.web.servlet.FrameworkServlet.doPut(FrameworkServlet.java:920)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:655)
    at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
    at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.ThreadLocalsCleanupFilter.doFilter(ThreadLocalsCleanupFilter.java:26)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:69)
    at org.geoserver.wms.animate.AnimatorFilter.doFilter(AnimatorFilter.java:70)
    at org.geoserver.filters.SpringDelegatingFilter$Chain.doFilter(SpringDelegatingFilter.java:66)
    at org.geoserver.filters.SpringDelegatingFilter.doFilter(SpringDelegatingFilter.java:41)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.platform.AdvancedDispatchFilter.doFilter(AdvancedDispatchFilter.java:37)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:320)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:70)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:127)
    at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:91)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:74)
    at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:70)
    at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:119)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:74)
    at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.geoserver.security.filter.GeoServerAnonymousAuthenticationFilter.doFilter(GeoServerAnonymousAuthenticationFilter.java:51)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:70)
    at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilterInternal(BasicAuthenticationFilter.java:215)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:74)
    at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
    at org.geoserver.security.filter.GeoServerBasicAuthenticationFilter.doFilter(GeoServerBasicAuthenticationFilter.java:81)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:70)
    at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:105)
    at org.geoserver.security.filter.GeoServerSecurityContextPersistenceFilter$1.doFilter(GeoServerSecurityContextPersistenceFilter.java:52)
    at org.geoserver.security.filter.GeoServerCompositeFilter$NestedFilterChain.doFilter(GeoServerCompositeFilter.java:74)
    at org.geoserver.security.filter.GeoServerCompositeFilter.doFilter(GeoServerCompositeFilter.java:91)
    at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:334)
    at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:215)
    at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:178)
    at org.geoserver.security.GeoServerSecurityFilterChainProxy.doFilter(GeoServerSecurityFilterChainProxy.java:142)
    at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:358)
    at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:271)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.LoggingFilter.doFilter(LoggingFilter.java:101)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:77)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.GZIPFilter.doFilter(GZIPFilter.java:53)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.SessionDebugFilter.doFilter(SessionDebugFilter.java:46)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.geoserver.filters.FlushSafeFilter.doFilter(FlushSafeFilter.java:42)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
    at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:119)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
    at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
    at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:374)
    at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
    at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888)
    at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1597)
    at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.MalformedURLException: no protocol: PKGo9m��!��ʏa?�Ɩ6�|9���Y��w��P��t���3��� ^O��(4a#���1��b���͸5H��C����$}�`'����Z�m9�(g%Nr���SJ�'�U:���J�7����yz�RN�A���PKG�I͵��`��I��R�����#T�Vc��e�S�|f��&I��92����x��na�_�����

[...Binary Data...]

    at java.base/java.net.URL.<init>(URL.java:645)
at java.base/java.net.URL.<init>(URL.java:541)
at java.base/java.net.URL.<init>(URL.java:488)
at org.geoserver.rest.util.RESTUtils.handleEXTERNALUpload(RESTUtils.java:201)
at org.geoserver.rest.catalog.AbstractStoreUploadController.handleFileUpload(AbstractStoreUploadController.java:80)

Tried with "Content-Type" instead of "Content-type". Error appears to be the same. GeoServer logs below:

2021-02-25 00:10:58,182 ERROR [geoserver.rest] - Error while storing uploaded file:
org.geoserver.rest.RestException 500 INTERNAL_SERVER_ERROR: Error while storing uploaded file:

[...]

Caused by: java.net.MalformedURLException: no protocol: PKGo9m��!��ʏa?�Ɩ6�|9���Y��w��P��t���3��� ^O��(4a#���1��b���͸5H��C����$}�`'����Z�m

[...]

at java.base/java.net.URL.<init>(URL.java:645)
at java.base/java.net.URL.<init>(URL.java:541)
at java.base/java.net.URL.<init>(URL.java:488)
at org.geoserver.rest.util.RESTUtils.handleEXTERNALUpload(RESTUtils.java:201)
at org.geoserver.rest.catalog.AbstractStoreUploadController.handleFileUpload(AbstractStoreUploadController.java:80)

Also tried using two different .zip just incase the error is related to my data. I created a new zip with a single SHP file and a new zip with a single Geotiff. Both produced the same error in the console:

> User-Agent: curl/7.61.1
> Accept: */*
> Content-Type: application/zip
> Content-Length: 4236196
> Expect: 100-continue
>
< HTTP/1.1 100
* We are completely uploaded and fine
< HTTP/1.1 500
< X-Frame-Options: SAMEORIGIN
< Content-Type: text/plain
< Transfer-Encoding: chunked
< Date: Thu, 25 Feb 2021 03:51:43 GMT
< Connection: close

So it still seems like Content-Type is not being detected correctly.

6
  • I would first check the GeoServer log file to see the whole error - it looks like the issue is with writing the files to storage not in the import.
    – Ian Turton
    Feb 23, 2021 at 8:39
  • Thanks Ian. I've added the log output to the question. Does my original command look right to you? If you have an example you could share of how to create a store from a zip that would be great.
    – Neil
    Feb 23, 2021 at 23:22
  • Can you try with Content-Type (with a capital T) rather than Content-type - I would have thought it would be ok but it might be the issue (as it clearly isn't recognising the content type)
    – Ian Turton
    Feb 24, 2021 at 8:44
  • Thanks - tried that. Error looks the same. I'll update question with new logs.
    – Neil
    Feb 25, 2021 at 0:14
  • Added results of a further test with new .zip files. Same error so I don't believe this is related to the data.
    – Neil
    Feb 25, 2021 at 3:56

1 Answer 1

0

Finally found the issue. When I was creating a zip of the shapefiles, one of the supporting files (.qix) was not being added to the zip. So while the zip was uploading successfully, GeoServer was unable to process the files to publish into layers.

1
  • 1
    I'm not sure that makes sense as a qix file is a QGIS add on to the standard and GeoServer doesn't/shouldn't need it.
    – Ian Turton
    Feb 26, 2021 at 9:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.