chore(molgenis-jenkins): Add molgenis-it pod template

molgenis-jenkins/README.md

@@ -40,57 +40,62 @@ You can use [all configuration values of the jenkins subchart](https://github.co
 ### GitHub Authentication delegation
 You need to setup a MOLGENIS - Jenkins GitHub OAuth App. You can do this by accessing this url: [add new OAuth app](https://github.com/settings/applications/new).
 
-### Additional configuration
-There is one additional group of configuration items specific for this chart, so not prefixed with `jenkins`:
+### Secrets
 
-* PipelineSecrets
-
-   When deployed, the chart creates a couple of kubernetes secrets that get used by jenkins and mounted in the jenkins 
-   build pods. The secrets, like the rest of the deployment, is namespaced so multiple instances can run beside
-   each other with their own secrets.
+   When deployed, the chart creates a couple of kubernetes secrets that get used by jenkins.
 
    You can override the values at deploy time but otherwise also configure them 
    [in Rancher](https://rancher.molgenis.org:7443/p/c-mhkqb:project-2pf45/secrets) or through kubectl.
 
-*  Vault
+#### Vault
 
-   New vault token to be used by the pods to retrieve their tokens from the vault.
+The vault secret gets mounted in the vault pod so pipeline scripts can retrieve secrets from the vault.
+
+| Parameter                 | Description                                | Default                                        |
+| ------------------------- | ------------------------------------------ | ---------------------------------------------- |
+| `secret.vault.token`      | Token to log into the hashicorp vault      | `xxxx`                                         |
+| `secret.vault.addr`       | Address of the vault                       | `https:vault-operator.vault-operator.svc:8200` |
+| `secret.vault.skipVerify` | Skip verification of the https connection  | `1`                                            |
+
+#### GitHub
+
+Token used by Jenkins to authenticate on GitHub.
+
+| Parameter             | Description              | Default            |
+| --------------------- | ------------------------ | ------------------ |
+| `secret.gitHub.user`  | username for the account | `molgenis-jenkins` |
+| `secret.gitHub.token` | token for the account    | `xxxx`             |
+
+#### Gogs
+
+Token used by Jenkins to authenticate on the [RuG Webhosting Gogs](https://git.webhosting.rug.nl).
+
+| Parameter           | Description              | Default   |
+| ------------------- | ------------------------ | --------- |
+| `secret.gogs.user`  | username for the account | `p281392` |
+| `secret.gogs.token` | token for the account    | `xxxx`    |
+
+#### Legacy:
+
+##### Docker Hub
    
-   | Parameter                          | Description                                | Default                                        |
-   | ---------------------------------- | ------------------------------------------ | ---------------------------------------------- |
-   | `PipelineSecrets.Vault.Replace`    | Replace the molgenis-pipeline-vault secret | `true`                                         |
-   | `PipelineSecrets.Vault.Token`      | Token to log into the hashicorp vault      | `xxxx`                                         |
-   | `PipelineSecrets.Vault.Addr`       | Address of the vault                       | `https:vault-operator.vault-operator.svc:8200` |
-   | `PipelineSecrets.Vault.SkipVerify` | Skip verification of the https connection  | `1`                                            |
+Account used in pipeline builds to push docker images to `hub.docker.com`.
+> They should read `secret/gcc/account/dockerhub` from vault instead!
 
-*  Env
+| Parameter                   | Description              | Default         |
+| --------------------------- | ------------------------ | --------------- |
+| `secret.dockerHub.user`     | username for the account | `molgenisci`    |
+| `secret.dockerHub.password` | password for the account | `xxxx`          |
+
+##### Registry
    
-   Environment variables stored in molgenis-pipeline-env secret, to be added as environment variables
-   in the slave pods.
+Account used in pipeline builds to push docker images to `registry.molgenis.org`.
+> They should read `secret/ops/account/nexus` from vault instead!
 
-   | Parameter                               | Description                               | Default         |
-   | --------------------------------------- | ----------------------------------------- | --------------- |
-   | `PipelineSecrets.Env.Replace`           | Replace molgenis-pipeline-env secret      | `true`          |
-   | `PipelineSecrets.Env.PGPPassphrase`     | passphrase for the pgp signing key        | `literal:xxxx`  |
-   | `PipelineSecrets.Env.CodecovToken`      | token for codecov.io                      | `xxxx`          |
-   | `PipelineSecrets.Env.GitHubToken`       | token for GH molgenis-jenkins user        | `xxxx`          |
-   | `PipelineSecrets.Env.NexusPassword`     | token for molgenis-jenkins user in NEXUS  | `xxxx`          |
-   | `PipelineSecrets.Env.DockerHubPassword` | token for molgenis user in hub.docker.com | `xxxx`          |
-   | `PipelineSecrets.Env.SonarToken`        | token for sonarcloud.io                   | `xxxx`          |
-   | `PipelineSecrets.Env.NpmToken`          | token for npmjs.org                       | `xxxx`          | 
-   | `PipelineSecrets.Env.SauceAccessKey`    | token for saucelabs.com                   | `xxxx`          |
-
-* File
-
-  Environment variables stored in molgenis-pipeline-file secret, to be mounted as files
-  in the `/root/.m2` directory of the slave pods.
-  > The settings.xml file references the 
-
-  | Parameter                              | Description                           | Default                                                                         |
-  | -------------------------------------- | ------------------------------------- | ------------------------------------------------------------------------------- |
-  | `PipelineSecrets.File.Replace`         | Replace molgenis-pipeline-file secret | `true`                                                                          |
-  | `PipelineSecrets.File.PGPPrivateKeyAsc`| pgp signing key in ascii form         | `-----BEGIN PGP PRIVATE KEY BLOCK-----xxxxx-----END PGP PRIVATE KEY BLOCK-----` |
-  | `PipelineSecrets.File.MavenSettingsXML`| Maven settings.xml file               | `<settings>[...]</settings>` (see actual [values.yaml](values.yaml))            |
+| Parameter                   | Description              | Default   |
+| --------------------------- | ------------------------ | --------- |
+| `secret.dockerHub.user`     | username for the account | `admin`   |
+| `secret.dockerHub.password` | password for the account | `xxxx`    |
 
 ## Command line use
 Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`.

molgenis-jenkins/resources/README.md

@@ -0,0 +1,6 @@
+To be able to run helm inside a jenkins pod, you'll need to 
+* create a role in the namespace where tiller is installed
+* bind that role to the user that jenkins pods run as
+
+This directory contains yaml for these resources.
+See also https://github.com/helm/helm/blob/master/docs/rbac.md

molgenis-jenkins/resources/jenkins-default-tiller-user-rolebinding.yaml

@@ -0,0 +1,13 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: RoleBinding
+metadata:
+  name: tiller-jenkins-binding
+  namespace: kube-system
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: Role
+  name: tiller-user
+subjects:
+- kind: ServiceAccount
+  name: default
+  namespace: molgenis-jenkins

molgenis-jenkins/resources/tiller-user-role.yaml

@@ -0,0 +1,18 @@
+apiVersion: rbac.authorization.k8s.io/v1beta1
+kind: Role
+metadata:
+  name: tiller-user
+  namespace: kube-system
+rules:
+- apiGroups:
+  - ""
+  resources:
+  - pods/portforward
+  verbs:
+  - create
+- apiGroups:
+  - ""
+  resources:
+  - pods
+  verbs:
+  - list

molgenis-jenkins/templates/molgenis-jenkins-dockerhub-secret.yaml

@@ -8,9 +8,9 @@ metadata:
     "jenkins.io/credentials-type": "usernamePassword"
   annotations: {
 # description - can not be a label as spaces are not allowed
-    "jenkins.io/credentials-description" : "user to authenticate against Docker Hub (hub.docker.com)"
+    "jenkins.io/credentials-description" : "(deprecated by vault) Account used in pipeline builds to push docker images to Docker Hub (hub.docker.com)"
   }
 type: Opaque
 data:
-  username: {{ "molgenisci" | b64enc | quote }}
-  password: {{ .Values.PipelineSecrets.Env.DockerHubPassword | b64enc | quote }}
+  username: {{ .Values.secret.registry.user | b64enc | quote }}
+  password: {{ .Values.secret.registry.password | b64enc | quote }}

molgenis-jenkins/templates/molgenis-jenkins-github-secret.yaml

@@ -8,9 +8,9 @@ metadata:
     "jenkins.io/credentials-type": "usernamePassword"
   annotations: {
 # description - can not be a label as spaces are not allowed
-    "jenkins.io/credentials-description" : "oauth token for the molgenis-jenkins github user"
+    "jenkins.io/credentials-description" : "Oauth token for the {{.Values.secret.gitHub.user}} GitHub user"
   }
 type: Opaque
 data:
-  username: {{ "molgenis-jenkins" | b64enc | quote }}
-  password: {{ .Values.PipelineSecrets.Env.GitHubToken | b64enc | quote }}
+  username: {{ .Values.secret.gitHub.user | b64enc | quote }}
+  password: {{ .Values.secret.gitHub.token | b64enc | quote }}

molgenis-jenkins/templates/molgenis-jenkins-gogs-secret.yaml

@@ -8,9 +8,9 @@ metadata:
     "jenkins.io/credentials-type": "usernamePassword"
   annotations: {
 # description - can not be a label as spaces are not allowed
-    "jenkins.io/credentials-description" : "user to authenticate against GOGS (git.webhosting.rug.nl)"
+    "jenkins.io/credentials-description" : "Account used to authenticate against RuG Webhosting Gogs."
   }
 type: Opaque
 data:
-  username: {{ "p281392" | b64enc | quote }}
-  password: {{ .Values.PipelineSecrets.Env.GogsToken | b64enc | quote }}
+  username: {{ .Values.secret.gogs.user | b64enc | quote }}
+  password: {{ .Values.secret.gogs.token | b64enc | quote }}

molgenis-jenkins/templates/molgenis-jenkins-nexus-secret.yaml

@@ -1,16 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-# this is the jenkins id.
-  name: "molgenis-jenkins-nexus-secret"
-  labels:
-# so we know what type it is.
-    "jenkins.io/credentials-type": "usernamePassword"
-  annotations: {
-# description - can not be a label as spaces are not allowed
-    "jenkins.io/credentials-description" : "user to authenticate against NEXUS"
-  }
-type: Opaque
-data:
-  username: {{ "admin" | b64enc | quote }}
-  password: {{ .Values.PipelineSecrets.Env.NexusPassword | b64enc | quote }}

molgenis-jenkins/templates/molgenis-jenkins-registry-secret.yaml

@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Secret
+metadata:
+  name: molgenis-jenkins-registry-secret
+  labels:
+    app: {{ template "jenkins.fullname" . }}
+    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
+    release: "{{ .Release.Name }}"
+    heritage: "{{ .Release.Service }}"
+  annotations: {
+# description - can not be a label as spaces are not allowed
+    "jenkins.io/credentials-description" : "(deprecated by vault) Account used in pipeline builds to push docker images to registry.molgenis.org."
+  }
+type: Opaque
+data:
+  username: {{ .Values.secret.registry.user | b64enc | quote }}
+  password: {{ .Values.secret.registry.password | b64enc | quote }}

molgenis-jenkins/templates/molgenis-jenkins-saucelabs-secret.yaml

@@ -1,16 +0,0 @@
-apiVersion: v1
-kind: Secret
-metadata:
-# this is the jenkins id.
-  name: "molgenis-jenkins-saucelabs-secret"
-  labels:
-# so we know what type it is.
-    "jenkins.io/credentials-type": "usernamePassword"
-  annotations: {
-# description - can not be a label as spaces are not allowed
-    "jenkins.io/credentials-description" : "user to authenticate against Saucelabs (saucelabs.com)"
-  }
-type: Opaque
-data:
-  username: {{ "molgenis-jenkins" | b64enc | quote }}
-  password: {{ .Values.PipelineSecrets.Env.SauceAccessKey | b64enc | quote }}

molgenis-jenkins/templates/molgenis-pipeline-env-secret.yaml

@@ -1,18 +0,0 @@
-{{- if .Values.PipelineSecrets.Env.Replace }}
-apiVersion: v1
-kind: Secret
-metadata:
-  name: molgenis-pipeline-env-secret
-  labels:
-    app: {{ template "jenkins.fullname" . }}
-    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
-    release: "{{ .Release.Name }}"
-    heritage: "{{ .Release.Service }}"
-type: Opaque
-data:
-  pgpPassphrase: {{ .Values.PipelineSecrets.Env.PGPPassphrase | b64enc | quote }}
-  codecovToken: {{ .Values.PipelineSecrets.Env.CodecovToken | b64enc | quote }}
-  githubToken: {{ .Values.PipelineSecrets.Env.GitHubToken | b64enc | quote }}
-  sonarToken: {{ .Values.PipelineSecrets.Env.SonarToken | b64enc | quote }}
-  npmToken: {{ .Values.PipelineSecrets.Env.NpmToken | b64enc | quote }}
-{{- end }}

molgenis-jenkins/templates/molgenis-pipeline-file-secret.yaml

@@ -1,15 +0,0 @@
-{{- if .Values.PipelineSecrets.File.Replace }}
-apiVersion: v1
-kind: Secret
-metadata:
-  name: molgenis-pipeline-file-secret
-  labels:
-    app: {{ template "jenkins.fullname" . }}
-    chart: "{{ .Chart.Name }}-{{ .Chart.Version }}"
-    release: "{{ .Release.Name }}"
-    heritage: "{{ .Release.Service }}"
-type: Opaque
-data:
-  key.asc: {{ .Values.PipelineSecrets.File.PGPPrivateKeyAsc | b64enc | quote }}
-  settings.xml: {{ .Values.PipelineSecrets.File.MavenSettingsXML | b64enc | quote }}
-{{- end }}

molgenis-jenkins/templates/molgenis-pipeline-vault-secret.yaml

@@ -1,4 +1,3 @@
-{{- if .Values.PipelineSecrets.Vault.Replace }}
 apiVersion: v1
 kind: Secret
 metadata:
@@ -10,7 +9,6 @@ metadata:
     heritage: "{{ .Release.Service }}"
 type: Opaque
 data:
-  token: {{ .Values.PipelineSecrets.Vault.Token | b64enc | quote }}
-  addr: {{ .Values.PipelineSecrets.Vault.Addr | b64enc | quote }}
-  skipVerify: {{ .Values.PipelineSecrets.Vault.SkipVerify | b64enc | quote }}
-{{- end }}
+  token: {{ .Values.secret.vault.token | b64enc | quote }}
+  addr: {{ .Values.secret.vault.addr | b64enc | quote }}
+  skipVerify: {{ .Values.secret.vault.skipVerify | b64enc | quote }}

molgenis-jenkins/values.yaml

@@ -368,8 +368,8 @@ jenkins:
     install: true
   Pods:
     molgenis:
-      Label: molgenisv2
-      NodeUsageMode: EXCLUSIVE
+      Label: molgenis
+      NodeUsageMode: NORMAL
       volumes:
         - type: HostPath
           hostPath: "/var/run/docker.sock"
@@ -416,39 +416,12 @@ jenkins:
               key: VAULT_ADDR
               secretName: molgenis-pipeline-vault-secret
               secretKey: addr
-      NodeSelector: {}
-    molgenis-legacy:
-      InheritFrom: molgenis
-      Label: molgenis
-      NodeUsageMode: NORMAL
-      volumes:
-        - type: Secret
-          secretName: molgenis-pipeline-file-secret
-          mountPath: "/home/jenkins/.m2"
-      Containers:
-      EnvVars:
-        - type: Secret
-          key: PGP_PASSPHRASE
-          secretName: molgenis-pipeline-env-secret
-          secretKey: pgpPassphrase
-        - type: KeyValue
-          key: PGP_SECRETKEY
-          value: "keyfile:/home.jenkins/.m2/key.asc"
-        - type: KeyValue
-          key: npm_config_registry
-          value: "http://nexus.molgenis-nexus:8081/repository/npm-central/"
-        - type: Secret
-          key: SONAR_TOKEN
-          secretName: molgenis-pipeline-env-secret
-          secretKey: sonarToken
-        - type: Secret
-          key: CODECOV_TOKEN
-          secretName: molgenis-pipeline-env-secret
-          secretKey: codecovToken
-        - type: Secret
-          key: GITHUB_TOKEN
-          secretName: molgenis-pipeline-env-secret
-          secretKey: githubToken
+        helm:
+          Image: "lachlanevenson/k8s-helm"
+          ImageTag: "v2.10.0"
+          Command: cat
+          WorkingDir: /home/jenkins
+          TTY: true
       NodeSelector: {}
     node:
       Label: node-carbon
@@ -461,22 +434,24 @@ jenkins:
           Command: cat
           WorkingDir: /home/jenkins
           TTY: true
-      EnvVars:
-        - type: KeyValue
-          key: npm_config_registry
-          value: "http://nexus.molgenis-nexus:8081/repository/npm-central/"
-        - type: Secret
-          key: CODECOV_TOKEN
-          secretName: molgenis-pipeline-env-secret
-          secretKey: codecovToken
-        - type: Secret
-          key: GITHUB_TOKEN
-          secretName: molgenis-pipeline-env-secret
-          secretKey: githubToken
-        - type: Secret
-          key: NPM_TOKEN
-          secretName: molgenis-pipeline-env-secret
-          secretKey: npmToken
+        vault:
+          Image: "vault"
+          Command: cat
+          WorkingDir: /home/jenkins
+          TTY: true
+          EnvVars:
+            - type: Secret
+              key: VAULT_TOKEN
+              secretName: molgenis-pipeline-vault-secret
+              secretKey: token
+            - type: Secret
+              key: VAULT_SKIP_VERIFY
+              secretName: molgenis-pipeline-vault-secret
+              secretKey: skipVerify
+            - type: Secret
+              key: VAULT_ADDR
+              secretName: molgenis-pipeline-vault-secret
+              secretKey: addr
       NodeSelector: {}
     molgenis-it:
       InheritFrom: molgenis
@@ -540,76 +515,27 @@ jenkins:
               cpu: "1"
               memory: "512Mi"
       NodeSelector: {}
-PipelineSecrets:
-  Vault:
-    Replace: true
-    Token: xxxx
-    Addr: "https://vault-operator.vault-operator.svc:8200"
-    SkipVerify: 1
-  Env:
-    # Set to false to keep existing secret
-    Replace: true
-    # Passphrase for the pgp private key file, prefixed with literal:
-    PGPPassphrase: literal:xxxx
-    # Token for codecov.io service
-    CodecovToken: xxxx
-    # Token for github bot account
-    GitHubToken: xxxx
-    # Token for github bot account
-    GogsToken: xxxx
-    # Token for sonarcloud.io
-    SonarToken: xxxx
-    # Token for npmjs.org
-    NpmToken: xxxx
-    # Password Local NEXUS
-    NexusPassword: xxxx
-    # Password hub.docker.com
-    DockerHubPassword: xxxx
-    # Access key for saucelabs.com
-    SauceAccessKey: xxxx
-  File:
-    # Set to false to keep existing secret
-    Replace: true
-    # PGP Private key in ascii format used to sign artifacts
-    PGPPrivateKeyAsc: |-
-      -----BEGIN PGP PRIVATE KEY BLOCK-----
-      xxxxx
-      -----END PGP PRIVATE KEY BLOCK-----
-    # maven.settings file
-    MavenSettingsXML: |-
-      <settings>
-        <localRepository>${user.home}/.mvnrepository</localRepository>
-        <interactiveMode>false</interactiveMode>
-        <mirrors>
-          <mirror>
-            <id>nexus</id>
-            <mirrorOf>external:*</mirrorOf>
-            <url>http://nexus.molgenis-nexus:8081/repository/maven-central/</url>
-          </mirror>
-        </mirrors>
-        <servers>
-          <!-- for snapshot builds of the master -->
-          <server>
-            <id>sonatype-nexus-staging</id>
-            <username>molgenis</username>
-            <password>xxxx</password>
-          </server>
-          <server>
-            <id>local-nexus</id>
-            <url>http://nexus.molgenis-nexus:8081/repository/maven-snapshots/</url>
-            <username>admin</username>
-            <password>xxxxx</password>
-          </server>
-          <!-- for docker images-->
-          <server>
-            <id>registry.molgenis.org</id>
-            <username>admin</username>
-            <password>xxxx</password>
-          </server>
-          <server>
-            <id>registry.hub.docker.com</id>
-            <username>molgenisci</username>
-            <password>xxxx</password>
-          </server>
-        </servers>
-      </settings>
+
+#secret contains configuration for the kubernetes secrets that jenkins can access
+secret:
+  # vault configures the vault secret
+  vault:
+    token: xxxx
+    addr: "https://vault-operator.vault-operator.svc:8200"
+    skipVerify: "1"
+  # githubToken contains access token for jenkins bot account on github.com
+  gitHub:
+    user: "molgenis-jenkins"
+    token: xxxx
+  # gogs contains access token for jenkins bot account on RuG GoGs
+  gogs:
+    user: p281392
+    token: xxxx
+  # registry contains credentials for registry.molgenis.org
+  registry:
+    user: admin
+    password: xxxx
+  # dockerHubPassword contains password for hub.docker.com
+  dockerHub:
+    user: molgenisci
+    password: xxxx

molgenis-preview/Chart.yaml

@@ -1,7 +1,7 @@
 apiVersion: v1
 appVersion: "1.0"
-description: A Helm chart for Kubernetes
-name: molgenis
+description: MOLGENIS - helm stack for testing purposes
+name: molgenis-preview
 version: 0.2.0
 sources:
 - https://git.webhosting.rug.nl/molgenis/molgenis-ops-docker-helm.git

molgenis-preview/README.md

@@ -1,5 +1,5 @@
 # MOLGENIS preview
-Is used for integration testing purposes.
+This chart is used for testing purposes. It can be used by data managers or developers to test MOLGENIS (e.g. integration testing).
 
 ## Containers
 This chart spins up a complete stack to run MOLGENIS. The created containers are:
@@ -9,3 +9,8 @@ This chart spins up a complete stack to run MOLGENIS. The created containers are
 - Elasticsearch
 - OpenCPU
 
+## Rancher
+You can spin up a test instance by navigating to https://rancher.molgenis.org:7777 and login with your LDAP-account.
+
+Go to the test-environment and click on "Launch". Search for MOLGENIS.
+

molgenis-preview/test.yaml

@@ -1,120 +0,0 @@
-# Source: molgenis/templates/deployment.yaml
-apiVersion: apps/v1beta2
-kind: Deployment
-metadata:
-  name: lanky-ragdoll-molgenis
-  labels:
-    app: molgenis
-    chart: molgenis-0.1.0
-    release: lanky-ragdoll
-    heritage: Tiller
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app: molgenis
-      release: lanky-ragdoll
-  template:
-    metadata:
-      labels:
-        app: molgenis
-        release: lanky-ragdoll
-    spec:
-      containers:
-        - name: molgenis
-          image: "registry.molgenis.org/molgenis/molgenis-app:latest"
-          imagePullPolicy: Always
-          env:
-            - name: molgenis.home
-              value: /home/molgenis
-            - name: opencpu.uri.host
-              value: opencpu
-            - name: elasticsearch.transport.addresses
-              value: elasticsearch:9300
-            - name: elasticsearch.cluster.name
-              value: molgenis
-            - name: db_uri
-              value: "jdbc:postgresql://postgres/molgenis"
-            - name: db_user
-              value: molgenis
-            - name: db_password
-              value: molgenis
-            - name: admin.password
-              value: admin
-            - name: CATALINA_OPTS
-              value: "-Xmx1g -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"
-          ports:
-            - containerPort: 8080
-          livenessProbe:
-            httpGet:
-              path: /
-              port: http
-          readinessProbe:
-            httpGet:
-              path: /api/v2/version
-              port: http
-          resources:
-            limits:
-              cpu: 1
-              memory: 1250Mi
-            requests:
-              cpu: 200m
-              memory: 1Gi
-
-
-        - name: elasticsearch
-          image: "docker.elastic.co/elasticsearch/elasticsearch:5.5.3"
-          imagePullPolicy: IfNotPresent
-          env:
-            - name: cluster.name
-              value: molgenis
-            - name: bootstrap.memory_lock
-              value: true
-            - name: ES_JAVA_OPTS
-              value: "-Xms512m -Xmx512m"
-            - name: xpack.security.enabled
-              value: false
-            - name: discovery.type
-              value: single-node
-          ports:
-            - containerPort: 9200
-            - containerPort: 9300
-            limits:
-              cpu: 1
-              memory: 1500Mi
-            requests:
-              cpu: 100m
-              memory: 1Gi
-
-
-        - name: postgres
-          image: "postgres:9.6-alpine"
-          imagePullPolicy: IfNotPresent
-          env:
-            - name: POSTGRES_USER
-              value: molgenis
-            - name: POSTGRES_PASSWORD
-              value: molgenis
-            - name: POSTGRES_DB
-              value: molgenis
-          ports:
-            - containerPort: 5432
-            limits:
-              cpu: 1
-              memory: 250Mi
-            requests:
-              cpu: 100m
-              memory: 250Mi
-
-
-        - name: opencpu
-          image: "molgenis/opencpu:latest"
-          imagePullPolicy: Always
-          ports:
-            - containerPort: 8004
-            limits:
-              cpu: 1
-              memory: 512Mi
-            requests:
-              cpu: 100m
-              memory: 256Mi

molgenis/Chart.yaml

@@ -0,0 +1,8 @@
+apiVersion: v1
+appVersion: "1.0"
+description: MOLGENIS - helm stack (in BETA)
+name: molgenis-beta
+version: 0.3.0
+sources:
+- https://git.webhosting.rug.nl/molgenis/molgenis-ops-docker-helm.git
+icon: https://git.webhosting.rug.nl/molgenis/molgenis-ops-docker-helm/raw/master/molgenis/catalogIcon-molgenis.svg

molgenis/README.md

@@ -0,0 +1,122 @@
+# MOLGENIS
+This chart is used for acceptance and production use cases.
+
+## Containers
+This chart spins up a MOLGENIS instance with HTTPD. The created containers are:
+
+- MOLGENIS
+
+## Provisioning
+You can choose from which registry you want to pull. There are 2 registries:
+- https://registry.molgenis.org
+- https://hub.docker.com
+
+The registry.molgenis.org contains the bleeding edge versions (PR's and master merges). The hub.docker.com contains the released artifacts (MOLGENIS releases and release candidates).
+
+The three properties you need to specify are:
+- ```molgenis.image.repository```
+- ```molgenis.image.name```
+- ```molgenis.image.tag```
+
+Besides determining which image you want to pull, you also have to set an administrator password. You can do this by specifying the following property. 
+- ```molgenis.adminPassword```
+
+## Services
+When you start MOLGENIS you need:
+- an elasticsearch instance (5.5.6) 
+- an postgres instance (9.6)
+
+You can attach additional services like:
+- an opencpu instance
+
+### Elasticsearch
+You can configure elasticsearch by giving in the cluster location.
+
+To configure the transport address you can address the node communication channel but also the native JAVA API. Which MOLGENIS uses to communicate with Elasticsearch.
+From Elasticsearch version 6 and further the JAVA API is not supported anymore. At this moment you can only use Elastic instance till major version 5.
+- ```molgenis.services.elasticsearch.transportAddresses: localhost:9300```
+
+To configure the index on a Elasticsearch cluster you can specify the clusterName property.
+- ```molgenis.services.elasticsearch.clusterName: molgenis```
+
+### Postgres
+You can specify the location of the postgres instance by specify the following property:
+- ```molgenis.services.postgres.host: localhost```
+
+You can specify the schema by filling out this property:
+- ```molgenis.services.postgres.scheme: molgenis```
+
+You can specify credentials for the database scheme by specifying the following properties:
+- ```molgenis.services.postgres.user: molgenis```
+- ```molgenis.services.postgres.password: molgenis```
+
+To test you can use the **PostgreSQL**-helm chart of Kubernetes and specify these answers:
+
+```bash
+# answers for postgresql chart
+postgresUser=molgenis
+postgresPassword=molgenis
+postgresDatabase=molgenis
+persistence.enabled=false
+```
+
+### OpenCPU
+You can specify the location of the OpenCPU cluster by specifying this property:
+- ```molgenis.services.opencpu.host: localhost```
+
+You can test OpenCPU settings using the **OpenCPU**-helm chart of MOLGENIS.
+
+## Resources
+You can specify resources by resource type. There are 2 resource types.
+- memory of container
+- maximum heap space JVM
+
+Specify memory usage of container:
+- ```molgenis.resources.limits.memory```
+
+Specify memory usage for Java JVM:
+- ```molgenis.javaOpts.maxHeapSpace```
+
+Select the resources you need dependant on the customer you need to serve.
+
+## Persistence
+You can enable persistence on your MOLGENIS stack by specifying the following property.
+
+- ```persistence.enabled```
+
+You can also choose to retain the volume of the NFS.
+- ```persistence.retain```
+
+The size and claim name can be specified per service. There are now two services that can be persist.
+
+- MOLGENIS
+- ElasticSearch
+
+MOLGENIS persistent properties.
+- ```molgenis.persistence.claim```
+- ```molgenis.persistence.size```
+
+ElasticSearch persistent properties.
+- ```elasticsearch.persistence.claim```
+- ```elasticsearch.persistence.size```
+
+
+### Resolve you persistent volume
+You do not know which volume is attached to your MOLGENIS instance. You can resolve this by executing:
+
+```
+kubectl get pv
+```
+
+You can now view the persistent volume claims and the attached volumes.
+
+| NAME | CAPACITY | ACCESS | MODES | RECLAIM | POLICY | STATUS | CLAIM | STORAGECLASS | REASON | AGE |
+| ---- | -------- | ------ | ----- | ------- | ------ | ------ | ----- | ------------ | ------ | --- |
+| pvc-45988f55-900f-11e8-a0b4-005056a51744 | 30G | RWX | | Retain | Bound | molgenis-solverd/molgenis-nfs-claim | nfs-provisioner-retain | | | 33d |
+| pvc-3984723d-220f-14e8-a98a-skjhf88823kk | 30G | RWO | | Delete | Bound | molgenis-test/molgenis-nfs-claim | nfs-provisioner | | | 33d |
+
+You see the ```molgenis-test/molgenis-nfs-claim``` is bound to the volume: ```pvc-3984723d-220f-14e8-a98a-skjhf88823kk```.
+When you want to view the data in the this volume you can go to the nfs-provisioning pod and execute the shell. Go to the directory ```export``` and lookup the directory ```pvc-3984723d-220f-14e8-a98a-skjhf88823kk```. 
+
+## Firewall
+Is defined at cluster level. This chart does not facilitate firewall configuration.

molgenis/questions.yml

@@ -0,0 +1,131 @@
+
+categories:
+- MOLGENIS
+questions:
+- variable: ingress.hosts[0].name
+  label: Hostname
+  default: "test.molgenis.org"
+  description: "Hostname for your stack"
+  type: hostname
+  required: true
+  group: "Load Balancing"
+- variable: molgenis.image.repository
+  label: Registry
+  default: "registry.hub.docker.com"
+  description: "Select a registry to pull from"
+  type: enum
+  options:
+  - "registry.hub.docker.com"
+  - "registry.molgenis.org"
+  required: true
+  group: "Provisioning"
+- variable: molgenis.image.tag
+  label: Version
+  default: ""
+  description: "Select a MOLGENIS version (check the registry.molgenis.org or hub.docker.com for released tags)"
+  type: string
+  required: true
+  group: "Provisioning"
+- variable: molgenis.adminPassword
+  label: Administrator password
+  default: ""
+  description: "Enter an administrator password"
+  type: password
+  required: true
+  group: "Provisioning"
+- variable: molgenis.services.opencpu.host
+  label: OpenCPU cluster
+  default: "localhost"
+  description: "Specify the OpenCPU cluster"
+  type: string
+  required: true
+  group: "Services"
+- variable: molgenis.services.postgres.host
+  label: Postgres cluster location
+  default: "postgresql.molgenis-postgresql.svc"
+  description: "Set the location of the postgres cluster"
+  type: string
+  required: true
+  group: "Services"
+- variable: molgenis.services.postgres.scheme
+  label: Database scheme
+  default: "molgenis"
+  description: "Set the database scheme"
+  type: string
+  required: true
+  group: "Services"
+- variable: molgenis.services.postgres.user
+  label: Database username
+  default: "molgenis"
+  description: "Set user of the database scheme"
+  type: string
+  required: true
+  group: "Services"
+- variable: molgenis.services.postgres.password
+  label: Database password
+  default: "molgenis"
+  description: "Set the password of the database scheme"
+  type: string
+  required: true
+  group: "Services"
+- variable: molgenis.resources.limits.memory
+  label: Container memory limit
+  default: 1250Mi
+  description: "Memory limit for this MOLGENIS container"
+  type: enum
+  options:
+  - "1250Mi"
+  - "2500Mi"
+  required: true
+  group: "Resources"
+- variable: molgenis.resources.requests.memory
+  label: Container memory reservation
+  default: 1250Mi
+  description: "Memory reservation for this MOLGENIS container (must fit in the selected memory limit for the container)"
+  type: enum
+  options:
+  - "1250Mi"
+  - "2500Mi"
+  required: true
+  group: "Resources"
+- variable: molgenis.javaOpts.maxHeapSpace
+  label: Maximum heap space (JVM)
+  default: "1g"
+  description: "Maximum heap space MOLGENIS container JVM. Please not this should fit in your container memory limit"
+  type: enum
+  options:
+  - "1g"
+  - "2g"
+  group: "Resources"
+- variable: persistence.enabled
+  default: false
+  description: "Do you want to use persistence"
+  type: boolean
+  required: true
+  group: "Persistence"
+  label: Persistence
+  show_subquestion_if: true
+  subquestions:
+  - variable: persistence.retain
+    default: false
+    description: "Do you want to retain the persistent volume"
+    type: boolean
+    label: Retain volume
+  - variable: molgenis.persistence.size
+    default: "30Gi"
+    description: "Size of MOLGENIS filestore (PostgreSQL and ElasticSearch excluded)"
+    type: enum
+    options:
+    - "30Gi"
+    - "50Gi"
+    - "100Gi"
+    label: Size MOLGENIS filestore
+  - variable: elasticsearch.persistence.size
+    default: "50Gi"
+    description: "Size of ElasticSearch data (directory that is persist: /usr/share/elasticsearch/data)"
+    type: enum
+    options:
+    - "50Gi"
+    - "100Gi"
+    - "200Gi"
+    label: Size for ElasticSearch data

molgenis/templates/NOTES.txt

@@ -0,0 +1,19 @@
+1. Get the application URL by running these commands:
+{{- if .Values.ingress.enabled }}
+{{- range .Values.ingress.hosts }}
+  http{{ if $.Values.ingress.tls }}s{{ end }}://{{ . }}{{ $.Values.ingress.path }}
+{{- end }}
+{{- else if contains "NodePort" .Values.service.type }}
+  export NODE_PORT=$(kubectl get --namespace {{ .Release.Namespace }} -o jsonpath="{.spec.ports[0].nodePort}" services {{ template "molgenis.fullname" . }})
+  export NODE_IP=$(kubectl get nodes --namespace {{ .Release.Namespace }} -o jsonpath="{.items[0].status.addresses[0].address}")
+  echo http://$NODE_IP:$NODE_PORT
+{{- else if contains "LoadBalancer" .Values.service.type }}
+     NOTE: It may take a few minutes for the LoadBalancer IP to be available.
+           You can watch the status of by running 'kubectl get svc -w {{ template "molgenis.fullname" . }}'
+  export SERVICE_IP=$(kubectl get svc --namespace {{ .Release.Namespace }} {{ template "molgenis.fullname" . }} -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
+  echo http://$SERVICE_IP:{{ .Values.service.port }}
+{{- else if contains "ClusterIP" .Values.service.type }}
+  export POD_NAME=$(kubectl get pods --namespace {{ .Release.Namespace }} -l "app={{ template "molgenis.name" . }},release={{ .Release.Name }}" -o jsonpath="{.items[0].metadata.name}")
+  echo "Visit http://127.0.0.1:8080 to use your application"
+  kubectl port-forward $POD_NAME 8080:80
+{{- end }}

molgenis/templates/_helpers.tpl

@@ -0,0 +1,32 @@
+{{/* vim: set filetype=mustache: */}}
+{{/*
+Expand the name of the chart.
+*/}}
+{{- define "molgenis.name" -}}
+{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+If release name contains chart name it will be used as a full name.
+*/}}
+{{- define "molgenis.fullname" -}}
+{{- if .Values.fullnameOverride -}}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- if contains $name .Release.Name -}}
+{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}
+
+{{/*
+Create chart name and version as used by the chart label.
+*/}}
+{{- define "molgenis.chart" -}}
+{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}}
+{{- end -}}

molgenis/templates/deployment.yaml

@@ -0,0 +1,126 @@
+apiVersion: apps/v1beta2
+kind: Deployment
+metadata:
+  {{- with .Values.ingress.annotations }}
+  annotations:
+{{ toYaml . | indent 4 }}
+  {{- end }}
+  name: {{ template "molgenis.fullname" . }}
+  labels:
+    app: {{ template "molgenis.name" . }}
+    chart: {{ template "molgenis.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  replicas: {{ .Values.replicaCount }}
+  selector:
+    matchLabels:
+      app: {{ template "molgenis.name" . }}
+      release: {{ .Release.Name }}
+  template:
+    metadata:
+      labels:
+        app: {{ template "molgenis.name" . }}
+        release: {{ .Release.Name }}
+    spec:
+      containers:
+        - name: molgenis
+        {{- with .Values.molgenis }}
+          image: {{ .image.repository }}/{{ .image.name }}:{{ .image.tag }}
+          imagePullPolicy: {{ .image.pullPolicy }}
+          env:
+            - name: molgenis.home
+              value: /home/molgenis
+            - name: opencpu.uri.host
+              value: {{ .services.opencpu.host }}
+            - name: elasticsearch.transport.addresses
+              value: {{ .services.elasticsearch.transportAddresses }}
+            - name: elasticsearch.cluster.name
+              value: {{ .services.elasticsearch.clusterName }}
+            - name: db_uri
+              value: jdbc:postgresql://{{ .services.postgres.host }}/{{ .services.postgres.scheme }}
+            - name: db_user
+              value: {{ .services.postgres.user }}
+            - name: db_password
+              value: {{ .services.postgres.password }}
+            - name: admin.password
+              value: {{ .adminPassword }}
+            - name: CATALINA_OPTS
+              value: "-Xmx{{ .javaOpts.maxHeapSpace }} -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled"
+          ports:
+            - containerPort: 8080
+          {{- if $.Values.persistence.enabled }}
+          volumeMounts:
+          - name: molgenis-nfs
+            mountPath: /home/molgenis
+          {{- end }}
+          livenessProbe:
+            httpGet:
+              path: /
+              port: 8080
+            initialDelaySeconds: 60
+            periodSeconds: 5
+            failureThreshold: 25
+            successThreshold: 1
+          readinessProbe:
+            httpGet:
+              path: /api/v2/version
+              port: 8080
+            initialDelaySeconds: 120
+            periodSeconds: 30
+            failureThreshold: 3
+            successThreshold: 1
+          resources:
+{{ toYaml .resources | indent 12 }}
+        {{- end }}
+
+        - name: elasticsearch
+        {{- with .Values.elasticsearch }}
+          image: "{{ .image.repository }}:{{ .image.tag }}"
+          imagePullPolicy: {{ .image.pullPolicy }}
+          env:
+            - name: cluster.name
+              value: {{ .clusterName }}
+            - name: bootstrap.memory_lock
+              value: "true"
+            - name: ES_JAVA_OPTS
+              value: "{{ .javaOpts }}"
+            - name: xpack.security.enabled
+              value: "false"
+            - name: discovery.type
+              value: single-node
+          ports:
+            - containerPort: 9200
+            - containerPort: 9300
+          {{- if $.Values.persistence.enabled }}
+          volumeMounts:
+          - name: elasticsearch-nfs
+            mountPath: /usr/share/elasticsearch/data
+          {{- end }}
+
+          resources:
+{{ toYaml .resources | indent 12 }}
+        {{- end }}
+
+{{- if .Values.persistence.enabled }}
+      volumes:
+        - name: molgenis-nfs
+          persistentVolumeClaim:
+            claimName: {{ .Values.molgenis.persistence.claim }}
+        - name: elasticsearch-nfs
+          persistentVolumeClaim:
+            claimName: {{ .Values.elasticsearch.persistence.claim }}
+{{- end }}
+
+    {{- with .Values.nodeSelector }}
+      nodeSelector:
+{{ toYaml . | indent 8 }}
+    {{- end }}
+    {{- with .Values.affinity }}
+      affinity:
+{{ toYaml . | indent 8 }}
+    {{- end }}
+    {{- with .Values.tolerations }}
+      tolerations:
+{{ toYaml . | indent 8 }}
+    {{- end }}

molgenis/templates/ingress.yaml

@@ -0,0 +1,38 @@
+{{- if .Values.ingress.enabled -}}
+{{- $fullName := include "molgenis.fullname" . -}}
+{{- $ingressPath := .Values.ingress.path -}}
+apiVersion: extensions/v1beta1
+kind: Ingress
+metadata:
+  name: {{ $fullName }}
+  labels:
+    app: {{ template "molgenis.name" . }}
+    chart: {{ template "molgenis.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+{{- with .Values.ingress.annotations }}
+  annotations:
+{{ toYaml . | indent 4 }}
+{{- end }}
+spec:
+{{- if .Values.ingress.tls }}
+  tls:
+  {{- range .Values.ingress.tls }}
+    - hosts:
+      {{- range .hosts }}
+        - {{ . }}
+      {{- end }}
+      secretName: {{ .secretName }}
+  {{- end }}
+{{- end }}
+  rules:
+  {{- range .Values.ingress.hosts }}
+    - host: {{ .name }}
+      http:
+        paths:
+          - path: {{ $ingressPath }}
+            backend:
+              serviceName: {{ $fullName }}
+              servicePort: 8080
+  {{- end }}
+{{- end }}

molgenis/templates/persistence/elasticsearchPVC.yaml

@@ -0,0 +1,19 @@
+{{- if .Values.persistence.enabled -}}
+apiVersion: extensions/v1beta1
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: {{ .Values.elasticsearch.persistence.claim }}
+  annotations:
+    {{- if .Values.persistence.retain }}
+    volume.beta.kubernetes.io/storage-class: "nfs-provisioner-retain"
+    {{- else }}
+    volume.beta.kubernetes.io/storage-class: "nfs-provisioner"
+    {{- end }}
+spec:
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: {{ .Values.elasticsearch.persistence.size }}
+{{- end }}

molgenis/templates/persistence/molgenisPVC.yaml

@@ -0,0 +1,19 @@
+{{- if .Values.persistence.enabled -}}
+apiVersion: extensions/v1beta1
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: {{ .Values.molgenis.persistence.claim }}
+  annotations:
+    {{- if .Values.persistence.retain }}
+    volume.beta.kubernetes.io/storage-class: "nfs-provisioner-retain"
+    {{- else }}
+    volume.beta.kubernetes.io/storage-class: "nfs-provisioner"
+    {{- end }}
+spec:
+  accessModes:
+    - ReadWriteMany
+  resources:
+    requests:
+      storage: {{ .Values.molgenis.persistence.size }}
+{{- end }}

molgenis/templates/service.yaml

@@ -0,0 +1,17 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{ template "molgenis.fullname" . }}
+  labels:
+    app: {{ template "molgenis.name" . }}
+    chart: {{ template "molgenis.chart" . }}
+    release: {{ .Release.Name }}
+    heritage: {{ .Release.Service }}
+spec:
+  type: {{ .Values.service.type }}
+  ports:
+    - name: molgenis
+      port: {{ .Values.service.port }}
+  selector:
+    app: {{ template "molgenis.name" . }}
+    release: {{ .Release.Name }}

molgenis/values.yaml

@@ -0,0 +1,77 @@
+# Default values for molgenis.
+
+replicaCount: 1
+
+service:
+  type: LoadBalancer
+  port: 8080
+
+ingress:
+  enabled: true
+  annotations:
+    nginx.ingress.kubernetes.io/proxy-body-size: "0"
+  path: /
+  hosts:
+  - name: test.molgenis.org
+  tls: []
+
+molgenis:
+  image:
+    repository: registry.hub.docker.com
+    name: molgenis/molgenis-app
+    tag: stable
+    pullPolicy: Always
+  adminPassword:
+  javaOpts:
+    maxHeapSpace: "1g"
+  resources:
+    limits:
+      cpu: 1
+      memory: 1250Mi
+    requests:
+      cpu: 200m
+      memory: 1250Mi
+  persistence:
+    claim: molgenis-nfs-claim
+    size: 30Gi
+  services:
+    opencpu:
+      host: localhost
+    elasticsearch:
+      transportAddresses: localhost:9300
+      clusterName: molgenis
+    postgres:
+      host: localhost
+      scheme: molgenis
+      user: molgenis
+      password: molgenis
+
+elasticsearch:
+  image:
+    repository: docker.elastic.co/elasticsearch/elasticsearch
+    tag: 5.5.3
+    pullPolicy: IfNotPresent
+  javaOpts: "-Xms1g -Xmx1g"
+  clusterName: molgenis
+  resources:
+   limits:
+    cpu: 2
+    memory: 3Gi
+   requests:
+    cpu: 100m
+    memory: 1Gi
+  persistence:
+    claim: elasticsearch-nfs-claim
+    size: 50Gi
+
+persistence:
+  enabled: false
+  retain: false
+
+nodeSelector: {
+  deployPod: "true"
+}
+
+tolerations: []
+
+affinity: {}