I am trying to run GeoNetwork 4.2.8 on Azure Kubernetes Service but am experiencing an issue with the catalogue-data not being persisted when the pod is restarted. In Kubernetes I have created a persistent volume that is set to retain the volume.
In the GeoNetwork AKS deployment I have set the environment variables:
env:
- name: DATA_DIR
value: /catalogue-data
The Java options look like this:
- name: JAVA_OPTS
value: |
-Dorg.eclipse.jetty.annotations.AnnotationParser.LEVEL=OFF -Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xms512M -Xss512M -Xmx2G -XX:+UseConcMarkSweepGC -Dgeonetwork.resources.dir=/catalogue-data/resources -Dgeonetwork.data.dir=/catalogue-data -Dgeonetwork.codeList.dir=/var/lib/jetty/webapps/geonetwork/WEB-INF/data/config/codelist -Dgeonetwork.schema.dir=/var/lib/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
As I understand it that should be it based on reading the geonetwork documentation: https://hub.docker.com/_/geonetwork/ However every time the pod restarts the file structure within catalogue-data is recreated. It appears geonetwork is formatting the volume and recreating the files structure each time. I have created other persistent volumes and mounted them on the geonetwork container and they are not impacted by restarts.
What is wrong? Obviously I want to avoid any data loss here but being quite new to GeoNetwork I am not entirely sure what the impact of losing the catalogue-data directory is. Is it recreated based on data in the database? Or should it absolutely be persisted?
Kubernetes details: Azure Kubernetes service Version: 1.28.5 Storage type: Azure disk
Storage class:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: azuredisk-csi-waitforfirstconsumer
provisioner: disk.csi.azure.com
parameters:
skuname: StandardSSD_LRS
allowVolumeExpansion: true
reclaimPolicy: Delete
volumeBindingMode: WaitForFirstConsumer
Persistent volume:
apiVersion: v1
kind: PersistentVolume
metadata:
annotations:
pv.kubernetes.io/provisioned-by: disk.csi.azure.com
name: {{ .Values.pv.geoCataloguePvName }}
spec:
capacity:
storage: {{ .Values.pv.geoCataloguePvSize }}
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
storageClassName: azuredisk-csi-waitforfirstconsumer
csi:
driver: disk.csi.azure.com
volumeHandle: {{ .Values.pv.geoCataloguePvDiskResourceId }}
volumeAttributes:
fsType: ext4
Persistent Volume Claim
{{- $global := .Values.global | default dict }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.pvc.geoCataloguePvcName }}
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: {{ .Values.pvc.geoCataloguePvcSize }}
volumeName: {{ .Values.pv.geoCataloguePvName }}
storageClassName: azuredisk-csi-waitforfirstconsumer
Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ include "geonetwork.fullname" . }}
annotations:
geonetwork.version: {{ .Chart.Version }}
labels:
{{ include "geonetwork.labels" . | indent 4 }}
spec:
replicas: {{ .Values.replicaCount }}
selector:
matchLabels:
geonetwork/app: {{ include "geonetwork.name" . }}
geonetwork/siteName: {{ .Release.Name }}
strategy:
type: Recreate
template:
metadata:
labels:
geonetwork/app: {{ include "geonetwork.name" . }}
geonetwork/siteName: {{ .Release.Name }}
spec:
{{- with .Values.imagePullSecrets }}
imagePullSecrets:
{{- toYaml . | nindent 8 }}
{{- end }}
securityContext:
runAsUser: 999
runAsGroup: 999
fsGroup: 999
containers:
- name: {{ .Chart.Name }}
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
imagePullPolicy: {{ .Values.image.pullPolicy }}
env:
- name: DATA_DIR
value: /catalogue-data
- name: ES_HOST
value: {{ .Values.elasticsearch.host }}
- name: ES_PORT
value: "{{ .Values.elasticsearch.port }}"
- name: ES_PROTOCOL
value: {{ .Values.elasticsearch.protocol }}
- name: GEONETWORK_DB_HOST
value: {{ .Values.postgres.host }}
- name: GEONETWORK_DB_NAME
value: {{ .Values.postgres.dbname }}
- name: GEONETWORK_DB_PASSWORD
value: {{ .Values.postgres.password }}
- name: GEONETWORK_DB_PORT
value: "{{ .Values.postgres.port }}"
- name: GEONETWORK_DB_TYPE
value: {{ .Values.db_type }}
- name: GEONETWORK_DB_USERNAME
value: {{ .Values.postgres.dbuser }}
- name: JAVA_OPTS
value: |
-Dorg.eclipse.jetty.annotations.AnnotationParser.LEVEL=OFF -Djava.security.egd=file:/dev/./urandom -Djava.awt.headless=true -Xms512M -Xss512M -Xmx2G -XX:+UseConcMarkSweepGC -Dgeonetwork.resources.dir=/catalogue-data/resources -Dgeonetwork.data.dir=/catalogue-data -Dgeonetwork.codeList.dir=/var/lib/jetty/webapps/geonetwork/WEB-INF/data/config/codelist -Dgeonetwork.schema.dir=/var/lib/jetty/webapps/geonetwork/WEB-INF/data/config/schema_plugins
ports:
- name: tcp
containerPort: 8080
protocol: TCP
resources:
{{- toYaml .Values.resources | nindent 12 }}
volumeMounts:
- mountPath: /catalogue-data
name: geonetwork
volumeMounts:
- mountPath: /metadata-import
name: geonetwork-metafileshare
restartPolicy: Always
volumes:
- name: geonetwork
persistentVolumeClaim:
claimName: {{ .Values.pvc.geoCataloguePvcName }}
- name: geonetwork-metafileshare
persistentVolumeClaim:
claimName: {{ .Values.pvc.geoMetaDataFileShareName }}
{{- with .Values.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.affinity }}
affinity:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
I do see this error in the pod when it first starts:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 56s default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
Warning FailedScheduling 54s default-scheduler 0/1 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/1 nodes are available: 1 Preemption is not helpful for scheduling..
Normal Scheduled 47s default-scheduler Successfully assigned geonetwork/geonetwork-758f99fd88-g8l59 to aks-nodepool1-12489523-vmss000001
Normal Pulled 46s kubelet Container image "jingking/geonetwork-hnap:4.2.8" already present on machine
Normal Created 45s kubelet Created container geonetwork-k8s
Normal Started 45s kubelet Started container geonetwork-k8s
Normal SuccessfulAttachVolume 34s attachdetach-controller AttachVolume.Attach succeeded for volume "geonetwork-azuredisk-pv"
