0

We have harvesting setup for several servers now. We're harvesting using one GeoNetwork v3.10.5 (Windows) and fetching from another GeoNetwork v3.10.0 (Linux). The Linux VM providing the data running CentOS 7 and configured with Tomcat. The Windows VM can see detect all of the catalogs, but lists them as unretrievable. The log files indicate that the directory "/public/metadata" is missing...which seems correct because I can't find it either.

Is there a configuration problem on the Linux VM?

2020-12-03T21:12:52,225+0000 INFO  [Amazonia] - Starting harvesting of Amazonia
2020-12-03T21:12:52,249+0000 INFO  [Amazonia] - Started harvesting from node : Amazonia (GeonetHarvester)
2020-12-03T21:12:52,254+0000 INFO  [Amazonia] - Retrieving information from : http://servir-data.ciat.cgiar.org/
2020-12-03T21:12:52,361+0000 INFO  [Amazonia] - Searching on Amazonia. From 1 to 2.
2020-12-03T21:12:52,519+0000 INFO  [Amazonia] - Client said maximum page size is 100
2020-12-03T21:12:52,519+0000 INFO  [Amazonia] - Page size returned by the server is greater than the default one for the harvester. Using the client harvester default page size (100)
2020-12-03T21:12:52,543+0000 INFO  [Amazonia] - Processing search with these parameters org.fao.geonet.kernel.harvest.harvester.geonet.Search@27fe5f6d[from=101,to=200,freeText=,title=,abstrac=,keywords=,digital=false,hardcopy=false,sourceUuid=,sourceName=,anyField=,anyValue=]
2020-12-03T21:12:52,543+0000 INFO  [Amazonia] - Searching on : Amazonia
2020-12-03T21:12:52,543+0000 INFO  [Amazonia] - Searching on Amazonia. From 1 to 100.
2020-12-03T21:12:52,587+0000 INFO  [Amazonia] - Total records processed from this search :8
2020-12-03T21:12:52,587+0000 INFO  [Amazonia] - Start of alignment for : Amazonia
2020-12-03T21:12:52,641+0000 ERROR [Amazonia] - /public/metadata
java.nio.file.NotDirectoryException: /public/metadata
    at com.sun.nio.zipfs.ZipDirectoryStream.<init>(ZipDirectoryStream.java:73)
    at com.sun.nio.zipfs.ZipPath.newDirectoryStream(ZipPath.java:651)
    at com.sun.nio.zipfs.ZipFileSystemProvider.newDirectoryStream(ZipFileSystemProvider.java:262)
    at java.nio.file.Files.newDirectoryStream(Unknown Source)
    at org.fao.geonet.utils.IO.isEmptyDir(IO.java:199)
    at org.fao.geonet.kernel.mef.MEF2Visitor.handleXml(MEF2Visitor.java:81)
    at org.fao.geonet.kernel.mef.MEF2Visitor.visit(MEF2Visitor.java:52)
    at org.fao.geonet.kernel.mef.MEFLib.visit(MEFLib.java:157)
    at org.fao.geonet.kernel.harvest.harvester.geonet.Aligner.addMetadata(Aligner.java:389)
    at org.fao.geonet.kernel.harvest.harvester.geonet.Aligner.align(Aligner.java:223)
    at org.fao.geonet.kernel.harvest.harvester.geonet.Harvester.harvest(Harvester.java:230)
    at org.fao.geonet.kernel.harvest.harvester.geonet.GeonetHarvester.doHarvest(GeonetHarvester.java:95)
    at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor.process(AbstractHarvester.java:602)
    at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:673)
    at org.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
2020-12-03T21:12:52,681+0000 ERROR [Amazonia] - /public/metadata
java.nio.file.NotDirectoryException: /public/metadata
    at com.sun.nio.zipfs.ZipDirectoryStream.<init>(ZipDirectoryStream.java:73)
    at com.sun.nio.zipfs.ZipPath.newDirectoryStream(ZipPath.java:651)
    at com.sun.nio.zipfs.ZipFileSystemProvider.newDirectoryStream(ZipFileSystemProvider.java:262)
    at java.nio.file.Files.newDirectoryStream(Unknown Source)
    at org.fao.geonet.utils.IO.isEmptyDir(IO.java:199)...

Here is the log from the Linux VM, the one being harvested.

2020-12-04 20:08:48,851 ERROR [jeeves.service] - Exception when executing service
2020-12-04 20:08:48,851 ERROR [jeeves.service] -  (C) Exc : java.lang.SecurityException: Current user can't download resources for metadata 'f8654dac-61ff-496f-a89b-a04919f074cc' and as such can't access the requested resource.
2020-12-04 20:08:48,852 ERROR [jeeves] - Error occurred within a transaction
java.lang.SecurityException: Current user can't download resources for metadata 'f8654dac-61ff-496f-a89b-a04919f074cc' and as such can't access the requested resource.
        at org.fao.geonet.api.records.attachments.AbstractStore.canDownload(AbstractStore.java:137)
        at org.fao.geonet.api.records.attachments.FilesystemStore.getResources(FilesystemStore.java:74)
        at org.fao.geonet.api.records.attachments.ResourceLoggerStore.getResources(ResourceLoggerStore.java:71)
        at org.fao.geonet.kernel.mef.MEF2Exporter.createMetadataFolder(MEF2Exporter.java:299)
        at org.fao.geonet.kernel.mef.MEF2Exporter.doExport(MEF2Exporter.java:215)
        at org.fao.geonet.kernel.mef.MEFLib.doMEF2Export(MEFLib.java:148)
        at org.fao.geonet.services.mef.Export.exec(Export.java:169)
        at jeeves.server.dispatchers.ServiceInfo.execService(ServiceInfo.java:227)
        at jeeves.server.dispatchers.ServiceInfo.noTransactionExec(ServiceInfo.java:142)
        at jeeves.server.dispatchers.ServiceInfo.access$000(ServiceInfo.java:46)
        at jeeves.server.dispatchers.ServiceInfo$1.doInTransaction(ServiceInfo.java:121)
        at jeeves.server.dispatchers.ServiceInfo$1.doInTransaction(ServiceInfo.java:118)
        at jeeves.transaction.TransactionManager.runInTransaction(TransactionManager.java:73)
        at jeeves.server.dispatchers.ServiceInfo.execServices(ServiceInfo.java:115)
        at jeeves.server.dispatchers.ServiceManager.dispatch(ServiceManager.java:449)
        at jeeves.server.dispatchers.ServiceManager.dispatch(ServiceManager.java:387)
        at jeeves.server.JeevesEngine.dispatch(JeevesEngine.java:593)
        at org.fao.geonet.services.main.GenericController.dispatch(GenericController.java:160)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:221)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:136)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:114)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:963)
        at jeeves.config.springutil.JeevesDispatcherServlet.access$101(JeevesDispatcherServlet.java:44)
        at jeeves.config.springutil.JeevesDispatcherServlet$1.doInTransaction(JeevesDispatcherServlet.java:56)
        at jeeves.config.springutil.JeevesDispatcherServlet$1.doInTransaction(JeevesDispatcherServlet.java:52)
        at jeeves.transaction.TransactionManager.runInTransaction(TransactionManager.java:73)
        at jeeves.config.springutil.JeevesDispatcherServlet.doDispatch(JeevesDispatcherServlet.java:49)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:897)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:861)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:624)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:731)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.fao.geonet.web.CORSResponseFilter.doFilter(CORSResponseFilter.java:129)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.fao.geonet.monitor.webapp.WebappMetricsFilter.doFilter(WebappMetricsFilter.java:121)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.fao.geonet.monitor.webapp.MetricsRegistryInitializerFilter.doFilter(MetricsRegistryInitializerFilter.java:58)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.fao.geonet.web.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:110)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite(RuleChain.java:176)
        at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java:145)
        at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest(UrlRewriter.java:92)
        at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter(UrlRewriteFilter.java:394)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.geonetwork.http.SessionTimeoutCookieFilter.doFilter(SessionTimeoutCookieFilter.java:91)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118)
        at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.session.SessionManagementFilter.doFilter(SessionManagementFilter.java:103)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.savedrequest.RequestCacheAwareFilter.doFilter(RequestCacheAwareFilter.java:45)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:199)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.authentication.www.BasicAuthenticationFilter.doFilter(BasicAuthenticationFilter.java:150)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at jeeves.config.springutil.PassthroughFilter.doFilter(PassthroughFilter.java:50)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.authentication.logout.LogoutFilter.doFilter(LogoutFilter.java:110)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.csrf.CsrfFilter.doFilterInternal(CsrfFilter.java:85)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87)
        at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342)
        at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192)
        at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160)
        at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
        at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
        at jeeves.config.springutil.JeevesDelegatingFilterProxy.doFilter(JeevesDelegatingFilterProxy.java:104)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
        at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
        at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
        at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
        at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:748)
9
  • In which version is the harvester, 3.10.0 or 3.10.5?
    – juanluisrp
    Dec 4, 2020 at 8:28
  • Is the log above from the Harvesting Windows GeoNetwork instance or are you looking at the Linux GeoNetwork logs for the harvest request?
    – nmtoken
    Dec 4, 2020 at 9:04
  • Is servir-data.ciat.cgiar.org the service being harvested? Note that that service does have a configuration error ~ gis.stackexchange.com/questions/379777/…
    – nmtoken
    Dec 4, 2020 at 9:05
  • The havester is v3.10.5.
    – jeffkn
    Dec 4, 2020 at 19:57
  • Yes, the Windows GeoNetwork is harvesting from the Linux GeoNetwork. The log above is from the Windows VM. I'll post the logs from the Linux VM in a bit.
    – jeffkn
    Dec 4, 2020 at 19:59

1 Answer 1

1

Seems that this is the issue that is causing the problem: https://github.com/geonetwork/core-geonetwork/issues/4425

I assume this may be resolved in v3.10.1 but have not yet verified.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.