diff --git a/molgenis-jenkins/README.md b/molgenis-jenkins/README.md index a2d1866..48c0a53 100644 --- a/molgenis-jenkins/README.md +++ b/molgenis-jenkins/README.md @@ -40,57 +40,62 @@ You can use [all configuration values of the jenkins subchart](https://github.co ### GitHub Authentication delegation You need to setup a MOLGENIS - Jenkins GitHub OAuth App. You can do this by accessing this url: [add new OAuth app](https://github.com/settings/applications/new). -### Additional configuration -There is one additional group of configuration items specific for this chart, so not prefixed with `jenkins`: +### Secrets -* PipelineSecrets - - When deployed, the chart creates a couple of kubernetes secrets that get used by jenkins and mounted in the jenkins - build pods. The secrets, like the rest of the deployment, is namespaced so multiple instances can run beside - each other with their own secrets. + When deployed, the chart creates a couple of kubernetes secrets that get used by jenkins. You can override the values at deploy time but otherwise also configure them [in Rancher](https://rancher.molgenis.org:7443/p/c-mhkqb:project-2pf45/secrets) or through kubectl. -* Vault +#### Vault - New vault token to be used by the pods to retrieve their tokens from the vault. +The vault secret gets mounted in the vault pod so pipeline scripts can retrieve secrets from the vault. + +| Parameter | Description | Default | +| ------------------------- | ------------------------------------------ | ---------------------------------------------- | +| `secret.vault.token` | Token to log into the hashicorp vault | `xxxx` | +| `secret.vault.addr` | Address of the vault | `https:vault-operator.vault-operator.svc:8200` | +| `secret.vault.skipVerify` | Skip verification of the https connection | `1` | + +#### GitHub + +Token used by Jenkins to authenticate on GitHub. + +| Parameter | Description | Default | +| --------------------- | ------------------------ | ------------------ | +| `secret.gitHub.user` | username for the account | `molgenis-jenkins` | +| `secret.gitHub.token` | token for the account | `xxxx` | + +#### Gogs + +Token used by Jenkins to authenticate on the [RuG Webhosting Gogs](https://git.webhosting.rug.nl). + +| Parameter | Description | Default | +| ------------------- | ------------------------ | --------- | +| `secret.gogs.user` | username for the account | `p281392` | +| `secret.gogs.token` | token for the account | `xxxx` | + +#### Legacy: + +##### Docker Hub - | Parameter | Description | Default | - | ---------------------------------- | ------------------------------------------ | ---------------------------------------------- | - | `PipelineSecrets.Vault.Replace` | Replace the molgenis-pipeline-vault secret | `true` | - | `PipelineSecrets.Vault.Token` | Token to log into the hashicorp vault | `xxxx` | - | `PipelineSecrets.Vault.Addr` | Address of the vault | `https:vault-operator.vault-operator.svc:8200` | - | `PipelineSecrets.Vault.SkipVerify` | Skip verification of the https connection | `1` | +Account used in pipeline builds to push docker images to `hub.docker.com`. +> They should read `secret/gcc/account/dockerhub` from vault instead! -* Env +| Parameter | Description | Default | +| --------------------------- | ------------------------ | --------------- | +| `secret.dockerHub.user` | username for the account | `molgenisci` | +| `secret.dockerHub.password` | password for the account | `xxxx` | + +##### Registry - Environment variables stored in molgenis-pipeline-env secret, to be added as environment variables - in the slave pods. +Account used in pipeline builds to push docker images to `registry.molgenis.org`. +> They should read `secret/ops/account/nexus` from vault instead! - | Parameter | Description | Default | - | --------------------------------------- | ----------------------------------------- | --------------- | - | `PipelineSecrets.Env.Replace` | Replace molgenis-pipeline-env secret | `true` | - | `PipelineSecrets.Env.PGPPassphrase` | passphrase for the pgp signing key | `literal:xxxx` | - | `PipelineSecrets.Env.CodecovToken` | token for codecov.io | `xxxx` | - | `PipelineSecrets.Env.GitHubToken` | token for GH molgenis-jenkins user | `xxxx` | - | `PipelineSecrets.Env.NexusPassword` | token for molgenis-jenkins user in NEXUS | `xxxx` | - | `PipelineSecrets.Env.DockerHubPassword` | token for molgenis user in hub.docker.com | `xxxx` | - | `PipelineSecrets.Env.SonarToken` | token for sonarcloud.io | `xxxx` | - | `PipelineSecrets.Env.NpmToken` | token for npmjs.org | `xxxx` | - | `PipelineSecrets.Env.SauceAccessKey` | token for saucelabs.com | `xxxx` | - -* File - - Environment variables stored in molgenis-pipeline-file secret, to be mounted as files - in the `/root/.m2` directory of the slave pods. - > The settings.xml file references the - - | Parameter | Description | Default | - | -------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------- | - | `PipelineSecrets.File.Replace` | Replace molgenis-pipeline-file secret | `true` | - | `PipelineSecrets.File.PGPPrivateKeyAsc`| pgp signing key in ascii form | `-----BEGIN PGP PRIVATE KEY BLOCK-----xxxxx-----END PGP PRIVATE KEY BLOCK-----` | - | `PipelineSecrets.File.MavenSettingsXML`| Maven settings.xml file | `[...]` (see actual [values.yaml](values.yaml)) | +| Parameter | Description | Default | +| --------------------------- | ------------------------ | --------- | +| `secret.dockerHub.user` | username for the account | `admin` | +| `secret.dockerHub.password` | password for the account | `xxxx` | ## Command line use Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. diff --git a/molgenis-jenkins/templates/molgenis-jenkins-dockerhub-secret.yaml b/molgenis-jenkins/templates/molgenis-jenkins-dockerhub-secret.yaml index e5a6214..5c35b46 100644 --- a/molgenis-jenkins/templates/molgenis-jenkins-dockerhub-secret.yaml +++ b/molgenis-jenkins/templates/molgenis-jenkins-dockerhub-secret.yaml @@ -8,9 +8,9 @@ metadata: "jenkins.io/credentials-type": "usernamePassword" annotations: { # description - can not be a label as spaces are not allowed - "jenkins.io/credentials-description" : "user to authenticate against Docker Hub (hub.docker.com)" + "jenkins.io/credentials-description" : "(deprecated by vault) Account used in pipeline builds to push docker images to Docker Hub (hub.docker.com)" } type: Opaque data: - username: {{ "molgenisci" | b64enc | quote }} - password: {{ .Values.PipelineSecrets.Env.DockerHubPassword | b64enc | quote }} \ No newline at end of file + username: {{ .Values.secret.registry.user | b64enc | quote }} + password: {{ .Values.secret.registry.password | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-jenkins-github-secret.yaml b/molgenis-jenkins/templates/molgenis-jenkins-github-secret.yaml index 2e7eba7..dc2b8df 100644 --- a/molgenis-jenkins/templates/molgenis-jenkins-github-secret.yaml +++ b/molgenis-jenkins/templates/molgenis-jenkins-github-secret.yaml @@ -8,9 +8,9 @@ metadata: "jenkins.io/credentials-type": "usernamePassword" annotations: { # description - can not be a label as spaces are not allowed - "jenkins.io/credentials-description" : "oauth token for the molgenis-jenkins github user" + "jenkins.io/credentials-description" : "Oauth token for the {{.Values.secret.gitHub.user}} GitHub user" } type: Opaque data: - username: {{ "molgenis-jenkins" | b64enc | quote }} - password: {{ .Values.PipelineSecrets.Env.GitHubToken | b64enc | quote }} \ No newline at end of file + username: {{ .Values.secret.gitHub.user | b64enc | quote }} + password: {{ .Values.secret.gitHub.token | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-jenkins-gogs-secret.yaml b/molgenis-jenkins/templates/molgenis-jenkins-gogs-secret.yaml index a16303d..ff6ae17 100644 --- a/molgenis-jenkins/templates/molgenis-jenkins-gogs-secret.yaml +++ b/molgenis-jenkins/templates/molgenis-jenkins-gogs-secret.yaml @@ -8,9 +8,9 @@ metadata: "jenkins.io/credentials-type": "usernamePassword" annotations: { # description - can not be a label as spaces are not allowed - "jenkins.io/credentials-description" : "user to authenticate against GOGS (git.webhosting.rug.nl)" + "jenkins.io/credentials-description" : "Account used to authenticate against RuG Webhosting Gogs." } type: Opaque data: - username: {{ "p281392" | b64enc | quote }} - password: {{ .Values.PipelineSecrets.Env.GogsToken | b64enc | quote }} \ No newline at end of file + username: {{ .Values.secret.gogs.user | b64enc | quote }} + password: {{ .Values.secret.gogs.token | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-jenkins-nexus-secret.yaml b/molgenis-jenkins/templates/molgenis-jenkins-nexus-secret.yaml deleted file mode 100644 index 4518e15..0000000 --- a/molgenis-jenkins/templates/molgenis-jenkins-nexus-secret.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: -# this is the jenkins id. - name: "molgenis-jenkins-nexus-secret" - labels: -# so we know what type it is. - "jenkins.io/credentials-type": "usernamePassword" - annotations: { -# description - can not be a label as spaces are not allowed - "jenkins.io/credentials-description" : "user to authenticate against NEXUS" - } -type: Opaque -data: - username: {{ "admin" | b64enc | quote }} - password: {{ .Values.PipelineSecrets.Env.NexusPassword | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-jenkins-registry-secret.yaml b/molgenis-jenkins/templates/molgenis-jenkins-registry-secret.yaml new file mode 100644 index 0000000..b853055 --- /dev/null +++ b/molgenis-jenkins/templates/molgenis-jenkins-registry-secret.yaml @@ -0,0 +1,17 @@ +apiVersion: v1 +kind: Secret +metadata: + name: molgenis-jenkins-registry-secret + labels: + app: {{ template "jenkins.fullname" . }} + chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" + release: "{{ .Release.Name }}" + heritage: "{{ .Release.Service }}" + annotations: { +# description - can not be a label as spaces are not allowed + "jenkins.io/credentials-description" : "(deprecated by vault) Account used in pipeline builds to push docker images to registry.molgenis.org." + } +type: Opaque +data: + username: {{ .Values.secret.registry.user | b64enc | quote }} + password: {{ .Values.secret.registry.password | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-jenkins-saucelabs-secret.yaml b/molgenis-jenkins/templates/molgenis-jenkins-saucelabs-secret.yaml deleted file mode 100644 index 382ba17..0000000 --- a/molgenis-jenkins/templates/molgenis-jenkins-saucelabs-secret.yaml +++ /dev/null @@ -1,16 +0,0 @@ -apiVersion: v1 -kind: Secret -metadata: -# this is the jenkins id. - name: "molgenis-jenkins-saucelabs-secret" - labels: -# so we know what type it is. - "jenkins.io/credentials-type": "usernamePassword" - annotations: { -# description - can not be a label as spaces are not allowed - "jenkins.io/credentials-description" : "user to authenticate against Saucelabs (saucelabs.com)" - } -type: Opaque -data: - username: {{ "molgenis-jenkins" | b64enc | quote }} - password: {{ .Values.PipelineSecrets.Env.SauceAccessKey | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-pipeline-env-secret.yaml b/molgenis-jenkins/templates/molgenis-pipeline-env-secret.yaml deleted file mode 100644 index ffdfd64..0000000 --- a/molgenis-jenkins/templates/molgenis-pipeline-env-secret.yaml +++ /dev/null @@ -1,18 +0,0 @@ -{{- if .Values.PipelineSecrets.Env.Replace }} -apiVersion: v1 -kind: Secret -metadata: - name: molgenis-pipeline-env-secret - labels: - app: {{ template "jenkins.fullname" . }} - chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" - release: "{{ .Release.Name }}" - heritage: "{{ .Release.Service }}" -type: Opaque -data: - pgpPassphrase: {{ .Values.PipelineSecrets.Env.PGPPassphrase | b64enc | quote }} - codecovToken: {{ .Values.PipelineSecrets.Env.CodecovToken | b64enc | quote }} - githubToken: {{ .Values.PipelineSecrets.Env.GitHubToken | b64enc | quote }} - sonarToken: {{ .Values.PipelineSecrets.Env.SonarToken | b64enc | quote }} - npmToken: {{ .Values.PipelineSecrets.Env.NpmToken | b64enc | quote }} -{{- end }} \ No newline at end of file diff --git a/molgenis-jenkins/templates/molgenis-pipeline-file-secret.yaml b/molgenis-jenkins/templates/molgenis-pipeline-file-secret.yaml deleted file mode 100644 index 65ac30b..0000000 --- a/molgenis-jenkins/templates/molgenis-pipeline-file-secret.yaml +++ /dev/null @@ -1,15 +0,0 @@ -{{- if .Values.PipelineSecrets.File.Replace }} -apiVersion: v1 -kind: Secret -metadata: - name: molgenis-pipeline-file-secret - labels: - app: {{ template "jenkins.fullname" . }} - chart: "{{ .Chart.Name }}-{{ .Chart.Version }}" - release: "{{ .Release.Name }}" - heritage: "{{ .Release.Service }}" -type: Opaque -data: - key.asc: {{ .Values.PipelineSecrets.File.PGPPrivateKeyAsc | b64enc | quote }} - settings.xml: {{ .Values.PipelineSecrets.File.MavenSettingsXML | b64enc | quote }} -{{- end }} diff --git a/molgenis-jenkins/templates/molgenis-pipeline-vault-secret.yaml b/molgenis-jenkins/templates/molgenis-pipeline-vault-secret.yaml index aa50fe7..da4598b 100644 --- a/molgenis-jenkins/templates/molgenis-pipeline-vault-secret.yaml +++ b/molgenis-jenkins/templates/molgenis-pipeline-vault-secret.yaml @@ -1,4 +1,3 @@ -{{- if .Values.PipelineSecrets.Vault.Replace }} apiVersion: v1 kind: Secret metadata: @@ -10,7 +9,6 @@ metadata: heritage: "{{ .Release.Service }}" type: Opaque data: - token: {{ .Values.PipelineSecrets.Vault.Token | b64enc | quote }} - addr: {{ .Values.PipelineSecrets.Vault.Addr | b64enc | quote }} - skipVerify: {{ .Values.PipelineSecrets.Vault.SkipVerify | b64enc | quote }} -{{- end }} \ No newline at end of file + token: {{ .Values.secret.vault.token | b64enc | quote }} + addr: {{ .Values.secret.vault.addr | b64enc | quote }} + skipVerify: {{ .Values.secret.vault.skipVerify | b64enc | quote }} \ No newline at end of file diff --git a/molgenis-jenkins/values.yaml b/molgenis-jenkins/values.yaml index a5ac886..6962082 100644 --- a/molgenis-jenkins/values.yaml +++ b/molgenis-jenkins/values.yaml @@ -368,8 +368,8 @@ jenkins: install: true Pods: molgenis: - Label: molgenisv2 - NodeUsageMode: EXCLUSIVE + Label: molgenis + NodeUsageMode: NORMAL volumes: - type: HostPath hostPath: "/var/run/docker.sock" @@ -417,39 +417,6 @@ jenkins: secretName: molgenis-pipeline-vault-secret secretKey: addr NodeSelector: {} - molgenis-legacy: - InheritFrom: molgenis - Label: molgenis - NodeUsageMode: NORMAL - volumes: - - type: Secret - secretName: molgenis-pipeline-file-secret - mountPath: "/home/jenkins/.m2" - Containers: - EnvVars: - - type: Secret - key: PGP_PASSPHRASE - secretName: molgenis-pipeline-env-secret - secretKey: pgpPassphrase - - type: KeyValue - key: PGP_SECRETKEY - value: "keyfile:/home.jenkins/.m2/key.asc" - - type: KeyValue - key: npm_config_registry - value: "http://nexus.molgenis-nexus:8081/repository/npm-central/" - - type: Secret - key: SONAR_TOKEN - secretName: molgenis-pipeline-env-secret - secretKey: sonarToken - - type: Secret - key: CODECOV_TOKEN - secretName: molgenis-pipeline-env-secret - secretKey: codecovToken - - type: Secret - key: GITHUB_TOKEN - secretName: molgenis-pipeline-env-secret - secretKey: githubToken - NodeSelector: {} node: Label: node-carbon NodeUsageMode: EXCLUSIVE @@ -461,155 +428,45 @@ jenkins: Command: cat WorkingDir: /home/jenkins TTY: true - EnvVars: - - type: KeyValue - key: npm_config_registry - value: "http://nexus.molgenis-nexus:8081/repository/npm-central/" - - type: Secret - key: CODECOV_TOKEN - secretName: molgenis-pipeline-env-secret - secretKey: codecovToken - - type: Secret - key: GITHUB_TOKEN - secretName: molgenis-pipeline-env-secret - secretKey: githubToken - - type: Secret - key: NPM_TOKEN - secretName: molgenis-pipeline-env-secret - secretKey: npmToken - NodeSelector: {} - molgenis-it: - InheritFrom: molgenis - Label: molgenis-it - NodeUsageMode: EXCLUSIVE - Containers: - elasticsearch: - Image: docker.elastic.co/elasticsearch/elasticsearch - ImageTag: 5.5.3 - resources: - requests: - cpu: "100m" - memory: "1Gi" - limits: - cpu: "1" - memory: "1500Mi" + vault: + Image: "vault" + Command: cat + WorkingDir: /home/jenkins + TTY: true EnvVars: - - type: KeyValue - key: ES_JAVA_OPTS - value: "-Xms512m -Xmx512m" - - type: KeyValue - key: cluster.name - value: molgenis - - type: KeyValue - key: bootstrap.memory_lock - value: "true" - - type: KeyValue - key: xpack.security.enabled - value: "false" - - type: KeyValue - key: discovery.type - value: single-node - postgres: - Image: postgres - ImageTag: 9.6-alpine - resources: - requests: - cpu: "100m" - memory: "250Mi" - limits: - cpu: "1" - memory: "250Mi" - EnvVars: - - type: KeyValue - key: POSTGRES_USER - value: molgenis - - type: KeyValue - key: POSTGRES_PASSWORD - value: molgenis - - type: KeyValue - key: POSTGRES_DB - value: molgenis - opencpu: - Image: molgenis/opencpu - AlwaysPullImage: true - resources: - requests: - cpu: "100m" - memory: "256Mi" - limits: - cpu: "1" - memory: "512Mi" + - type: Secret + key: VAULT_TOKEN + secretName: molgenis-pipeline-vault-secret + secretKey: token + - type: Secret + key: VAULT_SKIP_VERIFY + secretName: molgenis-pipeline-vault-secret + secretKey: skipVerify + - type: Secret + key: VAULT_ADDR + secretName: molgenis-pipeline-vault-secret + secretKey: addr NodeSelector: {} -PipelineSecrets: - Vault: - Replace: true - Token: xxxx - Addr: "https://vault-operator.vault-operator.svc:8200" - SkipVerify: 1 - Env: - # Set to false to keep existing secret - Replace: true - # Passphrase for the pgp private key file, prefixed with literal: - PGPPassphrase: literal:xxxx - # Token for codecov.io service - CodecovToken: xxxx - # Token for github bot account - GitHubToken: xxxx - # Token for github bot account - GogsToken: xxxx - # Token for sonarcloud.io - SonarToken: xxxx - # Token for npmjs.org - NpmToken: xxxx - # Password Local NEXUS - NexusPassword: xxxx - # Password hub.docker.com - DockerHubPassword: xxxx - # Access key for saucelabs.com - SauceAccessKey: xxxx - File: - # Set to false to keep existing secret - Replace: true - # PGP Private key in ascii format used to sign artifacts - PGPPrivateKeyAsc: |- - -----BEGIN PGP PRIVATE KEY BLOCK----- - xxxxx - -----END PGP PRIVATE KEY BLOCK----- - # maven.settings file - MavenSettingsXML: |- - - ${user.home}/.mvnrepository - false - - - nexus - external:* - http://nexus.molgenis-nexus:8081/repository/maven-central/ - - - - - - sonatype-nexus-staging - molgenis - xxxx - - - local-nexus - http://nexus.molgenis-nexus:8081/repository/maven-snapshots/ - admin - xxxxx - - - - registry.molgenis.org - admin - xxxx - - - registry.hub.docker.com - molgenisci - xxxx - - - \ No newline at end of file +#secret contains configuration for the kubernetes secrets that jenkins can access +secret: + # vault configures the vault secret + vault: + token: xxxx + addr: "https://vault-operator.vault-operator.svc:8200" + skipVerify: "1" + # githubToken contains access token for jenkins bot account on github.com + gitHub: + user: "molgenis-jenkins" + token: xxxx + # gogs contains access token for jenkins bot account on RuG GoGs + gogs: + user: p281392 + token: xxxx + # registry contains credentials for registry.molgenis.org + registry: + user: admin + password: xxxx + # dockerHubPassword contains password for hub.docker.com + dockerHub: + user: molgenisci + password: xxxx \ No newline at end of file diff --git a/molgenis/Chart.yaml b/molgenis/Chart.yaml index e60f8cc..d72e338 100644 --- a/molgenis/Chart.yaml +++ b/molgenis/Chart.yaml @@ -2,7 +2,7 @@ apiVersion: v1 appVersion: "1.0" description: MOLGENIS - helm stack (in BETA) name: molgenis-beta -version: 0.1.0 +version: 0.3.0 sources: - https://git.webhosting.rug.nl/molgenis/molgenis-ops-docker-helm.git icon: https://git.webhosting.rug.nl/molgenis/molgenis-ops-docker-helm/raw/master/molgenis/catalogIcon-molgenis.svg \ No newline at end of file diff --git a/molgenis/README.md b/molgenis/README.md index c85b200..03e8b2e 100644 --- a/molgenis/README.md +++ b/molgenis/README.md @@ -79,5 +79,44 @@ Specify memory usage for Java JVM: Select the resources you need dependant on the customer you need to serve. +## Persistence +You can enable persistence on your MOLGENIS stack by specifying the following property. + +- ```persistence.enabled``` + +You can also choose to retain the volume of the NFS. +- ```persistence.retain``` + +The size and claim name can be specified per service. There are now two services that can be persist. + +- MOLGENIS +- ElasticSearch + +MOLGENIS persistent properties. +- ```molgenis.persistence.claim``` +- ```molgenis.persistence.size``` + +ElasticSearch persistent properties. +- ```elasticsearch.persistence.claim``` +- ```elasticsearch.persistence.size``` + + +### Resolve you persistent volume +You do not know which volume is attached to your MOLGENIS instance. You can resolve this by executing: + +``` +kubectl get pv +``` + +You can now view the persistent volume claims and the attached volumes. + +| NAME | CAPACITY | ACCESS | MODES | RECLAIM | POLICY | STATUS | CLAIM | STORAGECLASS | REASON | AGE | +| ---- | -------- | ------ | ----- | ------- | ------ | ------ | ----- | ------------ | ------ | --- | +| pvc-45988f55-900f-11e8-a0b4-005056a51744 | 30G | RWX | | Retain | Bound | molgenis-solverd/molgenis-nfs-claim | nfs-provisioner-retain | | | 33d | +| pvc-3984723d-220f-14e8-a98a-skjhf88823kk | 30G | RWO | | Delete | Bound | molgenis-test/molgenis-nfs-claim | nfs-provisioner | | | 33d | + +You see the ```molgenis-test/molgenis-nfs-claim``` is bound to the volume: ```pvc-3984723d-220f-14e8-a98a-skjhf88823kk```. +When you want to view the data in the this volume you can go to the nfs-provisioning pod and execute the shell. Go to the directory ```export``` and lookup the directory ```pvc-3984723d-220f-14e8-a98a-skjhf88823kk```. + ## Firewall Is defined at cluster level. This chart does not facilitate firewall configuration. diff --git a/molgenis/questions.yml b/molgenis/questions.yml index 27dbee5..69c106a 100644 --- a/molgenis/questions.yml +++ b/molgenis/questions.yml @@ -81,7 +81,7 @@ questions: - variable: molgenis.resources.requests.memory label: Container memory reservation default: 1250Mi - description: "Memory reservation for this MOLGENIS container" + description: "Memory reservation for this MOLGENIS container (must fit in the selected memory limit for the container)" type: enum options: - "1250Mi" @@ -96,4 +96,36 @@ questions: options: - "1g" - "2g" - group: "Resources" \ No newline at end of file + group: "Resources" +- variable: persistence.enabled + default: false + description: "Do you want to use persistence" + type: boolean + required: true + group: "Persistence" + label: Persistence + show_subquestion_if: true + subquestions: + - variable: persistence.retain + default: false + description: "Do you want to retain the persistent volume" + type: boolean + label: Retain volume + - variable: molgenis.persistence.size + default: "30Gi" + description: "Size of MOLGENIS filestore (PostgreSQL and ElasticSearch excluded)" + type: enum + options: + - "30Gi" + - "50Gi" + - "100Gi" + label: Size MOLGENIS filestore + - variable: elasticsearch.persistence.size + default: "50Gi" + description: "Size of ElasticSearch data (directory that is persist: /usr/share/elasticsearch/data)" + type: enum + options: + - "50Gi" + - "100Gi" + - "200Gi" + label: Size for ElasticSearch data \ No newline at end of file diff --git a/molgenis/templates/deployment.yaml b/molgenis/templates/deployment.yaml index 9d4b356..b40a284 100644 --- a/molgenis/templates/deployment.yaml +++ b/molgenis/templates/deployment.yaml @@ -49,6 +49,11 @@ spec: value: "-Xmx{{ .javaOpts.maxHeapSpace }} -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled" ports: - containerPort: 8080 + {{- if $.Values.persistence.enabled }} + volumeMounts: + - name: molgenis-nfs + mountPath: /home/molgenis + {{- end }} livenessProbe: httpGet: path: / @@ -87,10 +92,25 @@ spec: ports: - containerPort: 9200 - containerPort: 9300 + {{- if $.Values.persistence.enabled }} + volumeMounts: + - name: elasticsearch-nfs + mountPath: /usr/share/elasticsearch/data + {{- end }} + resources: {{ toYaml .resources | indent 12 }} {{- end }} +{{- if .Values.persistence.enabled }} + volumes: + - name: molgenis-nfs + persistentVolumeClaim: + claimName: {{ .Values.molgenis.persistence.claim }} + - name: elasticsearch-nfs + persistentVolumeClaim: + claimName: {{ .Values.elasticsearch.persistence.claim }} +{{- end }} {{- with .Values.nodeSelector }} nodeSelector: diff --git a/molgenis/templates/persistence/elasticsearchPVC.yaml b/molgenis/templates/persistence/elasticsearchPVC.yaml new file mode 100644 index 0000000..6c8bef7 --- /dev/null +++ b/molgenis/templates/persistence/elasticsearchPVC.yaml @@ -0,0 +1,19 @@ +{{- if .Values.persistence.enabled -}} +apiVersion: extensions/v1beta1 +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: {{ .Values.elasticsearch.persistence.claim }} + annotations: + {{- if .Values.persistence.retain }} + volume.beta.kubernetes.io/storage-class: "nfs-provisioner-retain" + {{- else }} + volume.beta.kubernetes.io/storage-class: "nfs-provisioner" + {{- end }} +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: {{ .Values.elasticsearch.persistence.size }} +{{- end }} \ No newline at end of file diff --git a/molgenis/templates/persistence/molgenisPVC.yaml b/molgenis/templates/persistence/molgenisPVC.yaml new file mode 100644 index 0000000..6ec2f93 --- /dev/null +++ b/molgenis/templates/persistence/molgenisPVC.yaml @@ -0,0 +1,19 @@ +{{- if .Values.persistence.enabled -}} +apiVersion: extensions/v1beta1 +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: {{ .Values.molgenis.persistence.claim }} + annotations: + {{- if .Values.persistence.retain }} + volume.beta.kubernetes.io/storage-class: "nfs-provisioner-retain" + {{- else }} + volume.beta.kubernetes.io/storage-class: "nfs-provisioner" + {{- end }} +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: {{ .Values.molgenis.persistence.size }} +{{- end }} \ No newline at end of file diff --git a/molgenis/values.yaml b/molgenis/values.yaml index 86d3cc1..44b9f27 100644 --- a/molgenis/values.yaml +++ b/molgenis/values.yaml @@ -25,12 +25,15 @@ molgenis: javaOpts: maxHeapSpace: "1g" resources: - limits: - cpu: 1 - memory: 1250Mi - requests: - cpu: 200m - memory: 1250Mi + limits: + cpu: 1 + memory: 1250Mi + requests: + cpu: 200m + memory: 1250Mi + persistence: + claim: molgenis-nfs-claim + size: 30Gi services: opencpu: host: localhost @@ -57,8 +60,17 @@ elasticsearch: requests: cpu: 100m memory: 1Gi + persistence: + claim: elasticsearch-nfs-claim + size: 50Gi -nodeSelector: {} +persistence: + enabled: false + retain: false + +nodeSelector: { + deployPod: "true" +} tolerations: []