Skip to content

Commit 1dfcc91

Browse files
6l206l20
authored
Feature/qkm deploy #628 (#47)
* Added qkm + qkm-postgresql * Add charts default + ability to disable QKM postgres * Test is disabled * Condition on Vault CSR * Vault default is disabled * default orchestrate version is 21.1.12 * Update default repo * Qkm default is false * Add conditions on charts * Remove vault operator from charts * Add qkm related values * Add new env vars * Fix Ingresses * Add migration related env vars * Map values * Add qkm values * Change vault default image * Remove initContainers part * Enable vault + qkm * Enable one agent as sample * fix azure issue * Updated common values * Disable kafka in default * Update staging profile * Make kafka conditionnal * fix missing name on azure * Let api key file in values * clean up DB vars * Make kafka chart conditionnal * Use env vars for postgres * Remove hard coded values * missing username * Addressed Dario's comment * Align Vault plugin values * Remove useless intDb script * Allow private repo for qkm * Added overview picture * Updated default values * Updated CHANGELOG * Add qkm api-key sample * Update Readme with api-keys note * overview image reviewed
1 parent bc57727 commit 1dfcc91

15 files changed

+438
-117
lines changed

‎CHANGELOG.md‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,16 @@
22

33
All notable changes to this project will be documented in this file.
44

5+
## v6.0.0 (Unreleased)
6+
7+
* Support latest v21.12.x orchestrate version
8+
* Support Hashicorp Vault image with embedded plugin
9+
* Remove Key manager and replace it with Quorum Key Manager dependency
10+
* Make chart Azure compliant
11+
* Updated Ingresses
12+
* Made dependencies optional
13+
14+
515
## v5.1.0 (Unreleased)
616

717
* Make Vault, Postgres, Redis Highly Available in Multi-Availability Zones

‎README.md‎

Lines changed: 41 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ For more information, refer to the [Orchestrate documentation](https://docs.orch
1515
<H1>Orchestrate-Kubernetes</H1>
1616

1717
- [Codefi Orchestrate](#codefi-orchestrate)
18+
- [Deployment overview](#overview)
1819
- [Compatibility](#compatibility)
1920
- [1. Requirements](#1-requirements)
2021
- [1.1. CLI tools](#11-cli-tools)
@@ -32,11 +33,20 @@ For more information, refer to the [Orchestrate documentation](https://docs.orch
3233

3334
This repository contains an implementation example on how to deploy Orchestrate and its dependencies using Kubernetes and Helm charts.
3435

36+
# Overview
37+
38+
Below is a high level diagram of what this chart will help you deploy
39+
40+
41+
![overview diagram](./img/overview.png "Overview")
42+
43+
3544
# Compatibility
3645

3746
| Orchestrate-kubernetes versions | Orchestrate versions |
3847
|---------------------------------|-------------------------------|
39-
| master/HEAD | Orchestrate v21.1.x or higher |
48+
| master/HEAD | Orchestrate v21.12.x or higher |
49+
| v6.0.0 | Orchestrate v21.12.x or higher |
4050
| v5.0.0 | Orchestrate v21.1.x or higher |
4151
| v4.0.0 | Orchestrate v2.5.x |
4252
| v3.1.0 | Orchestrate v2.5.x |
@@ -120,18 +130,23 @@ This repository provides few examples of environment values sets:
120130

121131
Note: All the passwords and usernames of every dependendcies are located in `environments/common.yaml.gotmpl`. Do not forget to change, eventually extract, those values depending on how you want to manage those secrets.
122132

133+
Note: The ./values/api-key/api-keys.csv gives an exemple of what you should use for the Quorum Key Manager api-keys when this mode is enabled, provided values MUST be changed in a Prod environment.
134+
123135
The following tables lists the configurable values for the environments. Some of them are directly configurable bia envronement variable:
124136

125137
| Parameter | Description | Default |
126138
|------------------------------------------------|----------------------------------------------------------------------------|-------------------------------------------------------------|
127139
| `orchestrate.namespace` | Namespace where Orchestrate will be deployed (env `ORCHESTRATE_NAMESPACE`) | `orchestrate` |
140+
| `orchestrate.chart.name` | This deployment orchestrate chart (env `ORCHESTRATE_CHART`) | `consensys/orchestrate` |
141+
| `orchestrate.chart.version` | Namespace where Orchestrate will be deployed (env `ORCHESTRATE_CHART_VERSION`) | `1.0.6` |
128142
| `orchestrate.global.imageCredentials.registry` | Docker registry where Orchestrate images are stored (env `REGISTRY_URL`) | `docker.consensys.net` |
129143
| `orchestrate.global.imageCredentials.username` | [REQUIRED] Username of the registry (env `REGISTRY_USERNAME`) | |
130144
| `orchestrate.global.imageCredentials.password` | [REQUIRED] Password of the registry (env `REGISTRY_PASSWORD`) | |
131145
| `orchestrate.global.image.repository` | Path to Orchestrate image (env `ORCHESTRATE_REPOSITORY`) | `docker.consensys.net/priv/orchestrate` |
132146
| `orchestrate.global.image.tag` | Orchestrate image tag (env `ORCHESTRATE_TAG`) | `v21.1.2` |
133147
| `orchestrate.api` | Orchestrate API values | |
134-
| `orchestrate.keyManager` | Orchestrate Key Manager values | |
148+
| `orchestrate.keyManager` | Orchestrate Key Manager values, for usage with version 21.1.X | |
149+
| `orchestrate.qkm` | Orchestrate Key Manager values, for usage with version 21.10.X | |
135150
| `orchestrate.txListener` | Orchestrate Tx Listener values | `nil` |
136151
| `orchestrate.txSender` | Orchestrate Tx Sender values | `nil` |
137152
| `orchestrate.test.image.repository` | Path to Orchestrate test image (env `TEST_REPOSITORY`) | `nil` |
@@ -150,13 +165,14 @@ For more information about Vault Operator, please see https://github.com/banzaic
150165
|-----------------------|------------------------------------------------------------------------------------|--------------------------------------------------------------------|
151166
| `vault.namespace` | Namespace where Hashicop Vault will be deployed (env `VAULT_NAMESPACE`) | `orchestrate` |
152167
| `vault.replicaCount` | Number of Vault instance | `1` |
153-
| `vault.plugin.tag` | Orchestrate Hashicorp Vault Plugin tag (env `VAULT_PLUGIN_TAG`) | `v0.0.9` |
154-
| `vault.plugin.sha256` | Orchestrate Hashicorp Vault Plugin SHA256 checksum (env `VAULT_PLUGIN_SHA256SUM`) | `4919a7fcf66fe98b459e6a46f9233aae9fc2f224ccbb6a44049e2f608b9eebf5` |
168+
| `vault.plugin.tag` | Orchestrate Hashicorp Vault Plugin tag (env `VAULT_PLUGIN_TAG`) | `v1.1.3` |
169+
| `vault.plugin.sha256` | Orchestrate Hashicorp Vault Plugin SHA256 checksum (env `VAULT_PLUGIN_SHA256SUM`) | `e084800c61749a9c7b51f6e91bb89ab6d5a2678cdb707eaa73f9bef0cf73fc61` |
155170

156171
For more information about values defined in values/vault.yaml.gotmpl, please see https://github.com/banzaicloud/bank-vaults/tree/master/operator/deploy and https://github.com/banzaicloud/bank-vaults/tree/master/charts/vault
157172

158173
| Parameter | Description | Default |
159174
|---------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------|
175+
| `kafka.enabled` | Use to enable or disable the kafka deployment, might be replaced by an existin EventHub on Azure for instance | `true` |
160176
| `kafka.namespace` | Namespace where Kafka and Zookeeper Vault will be deployed (env `KAFKA_NAMESPACE`) | `1` |
161177
| `kafka.replicaCount` | Number of Kafka instance nodes | `1` |
162178
| `kafka.numPartitions` | The default number of log partitions per topic | `1` |
@@ -230,9 +246,25 @@ For more information about values defined in values/postgresql.yaml.gotmpl, plea
230246
|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------|
231247
| `domainName` | (Option) Domain name registered to the ingress controller of your kubernetes cluster. If not empty Orchestrate API will be exposed to {{orchestrate.namespace}}.{{domainName}}. If the observability stack is enabled grafana.{{domainName}} and prometheus.{{domainName}} will be exposed too (env `DOMAIN_NAME`) | `` |
232248

249+
250+
Values below are useful when deploying orchestrate with version 21.10.X, having possibly a Quorum Key Manager running independently
251+
252+
| Parameter | Description | Default |
253+
|----------------------------|-----------------------------------------------------------------------|---------------|
254+
| `qkm.enabled` | If true, Quorum Key Manager will be deployed | `true` |
255+
| `qkm.url` | Url where Quorum Key Manager may be reached (env `QKM_URL`) | `http://quorumkeymanager.orchestrate` |
256+
| `qkm.namespace` | Namespace where Quorum Key Manager is deployed (env `QKM_NAMESPACE`) | `orchestrate` |
257+
| `qkm.orchestrate.storeName`| Initial and existing eth-account name used by orchestrate | `eth-accounts` |
258+
| `qkm.orchestrate.apiKey` | Existing apiKey used by orchestrate to authenticate | `YWRtaW4tdXNlcg==` |
259+
| `qkm.chart.name` | Helm chart of your Quorum Key Manager deployment | `consensys/quorumkeymanager` |
260+
| `qkm.chart.version` | Helm chart version of your Quorum Key Manager deployment | `1.1.1` |
261+
| `qkm.port` | Port of the Quorum Key Manager service | `8080` |
262+
263+
For more information about values defined in values/qkm.yaml.gotmpl, please refer to https://github.com/ConsenSys/quorum-key-manager-helm
264+
233265
# 3. Hashicorp Vault
234266

235-
This helmfiles deploys [Hashicorp's Vault](https://www.vaultproject.io/) with integrated storage with raft with [Bank-Vaults](https://github.com/banzaicloud/bank-vaults). We deploy first the Vault operator, then the following ressources contained in `values/vault.yaml`:
267+
This helmfiles optionally deploys [Hashicorp's Vault](https://www.vaultproject.io/) with integrated storage with raft with [Bank-Vaults](https://github.com/banzaicloud/bank-vaults). We deploy first the Vault operator, then the following ressources contained in `values/vault.yaml`:
236268
- Vault CRD's, including [Vault policy](https://www.vaultproject.io/docs/concepts/policies), [Vault authentication](https://www.vaultproject.io/docs/concepts/auth), and [Orchestrate Hashicorp Vault Plugin](https://github.com/ConsenSys/orchestrate-hashicorp-vault-plugin)
237269

238270
[Vault policy](https://www.vaultproject.io/docs/concepts/policies)
@@ -274,6 +306,8 @@ This helmfiles deploys [Hashicorp's Vault](https://www.vaultproject.io/) with in
274306
- Service Account
275307
- RBAC configuration
276308

309+
Note that it is highly recommended to use the `consensys/quorum-hashicorp-vault-plugin` image when deplying a Vault ressource.
310+
277311
# 4. Observability
278312

279313
This helmfile could deploy [Prometheus Operator](https://github.com/coreos/prometheus-operator) and [Prometheus](https://prometheus.io/) based on the [Kube-Prometheus Helm chart](https://github.com/bitnami/charts/tree/master/bitnami/kube-prometheus). It also deploys Grafana with default dashboards for Orchestrate, Kubernetes, Golang, Kafka, Postgres, Redis, and Hashicorp Vault
@@ -295,4 +329,5 @@ kubectl port-forward --namespace $OBSERVABILITY_NAMESPACE svc/grafana 3000:80
295329

296330
## 5.1. From Orchestrate v2.5.X to v21.1.X
297331

298-
[Read the steps to upgrade Orchestrate v2.5.X to v21.1.X](docs/upgrades/v21-1-X.md)
332+
[Read the steps to upgrade Orchestrate v2.5.X to v21.1.X](docs/upgrades/v21-1-X.md)
333+

‎environments/common.yaml.gotmpl‎

Lines changed: 114 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,33 @@
11
---
22
{{ $defaultNamespace := "orchestrate" }}
3-
{{ $tag := "v21.1.3" }}
3+
{{ $defaultOrchestrateChart := "consensys/orchestrate" }}
4+
{{ $defaultQkmChart := "consensys/quorumkeymanager" }}
5+
{{ $defaultOrchestrateChartVersion := "2.0.0" }}
6+
{{ $defaultQkmChartVersion := "1.1.5" }}
7+
{{ $defaultNamespace := "orchestrate" }}
8+
{{/* $tag := "v21.10.1-alpha.3" */}}
9+
{{ $tag := "v21.12.1" }}
10+
{{ $qkmTag := "v21.12.1" }}
411

512
orchestrate:
613
namespace: {{ env "ORCHESTRATE_NAMESPACE" | default $defaultNamespace }}
14+
chart:
15+
name: {{ env "ORCHESTRATE_CHART" | default $defaultOrchestrateChart }}
16+
version: {{ env "ORCHESTRATE_CHART_VERSION" | default $defaultOrchestrateChartVersion }}
717

818
global:
919
imageCredentials:
1020
registry: {{ env "REGISTRY_URL" | default "docker.consensys.net" }}
1121
username: {{ requiredEnv "REGISTRY_USERNAME" }}
1222
password: {{ requiredEnv "REGISTRY_PASSWORD" }}
1323
image:
14-
repository: {{ env "ORCHESTRATE_REPOSITORY" | default "docker.consensys.net/priv/orchestrate" }}
24+
repository: {{ env "ORCHESTRATE_REPOSITORY" | default "consensys/orchestrate" }}
1525
tag: {{ env "ORCHESTRATE_TAG" | default $tag | quote }}
1626
environment: {}
1727
environmentSecrets: {}
1828

1929
api:
20-
environment: {}
21-
environmentSecrets: {}
22-
23-
keyManager:
30+
enabled: true
2431
environment: {}
2532
environmentSecrets: {}
2633

@@ -31,17 +38,36 @@ orchestrate:
3138
txSender:
3239
environment: {}
3340
environmentSecrets: {}
41+
42+
migrate:
43+
environment: {}
44+
environmentSecrets: {}
45+
46+
47+
auth:
48+
jwt:
49+
issuerUrl: https://consensys.eu.auth0.com
50+
audience: https://orchestrate.consensys.net
51+
claims: https://api.orchestrate.network
3452

3553
vaultOperator:
54+
enabled: false
3655
namespace: {{ env "VAULT_OPERATOR_NAMESPACE" | default "vault-operator" }}
3756

3857
vault:
58+
enabled: true
59+
replicaCount: 1
3960
namespace: {{ env "VAULT_NAMESPACE" | default ( env "ORCHESTRATE_NAMESPACE" | default $defaultNamespace ) }}
4061
plugin:
41-
tag: {{ env "VAULT_PLUGIN_TAG" | default "v0.0.9" }}
42-
sha256: {{ env "VAULT_PLUGIN_SHA256SUM" | default "4919a7fcf66fe98b459e6a46f9233aae9fc2f224ccbb6a44049e2f608b9eebf5" }}
62+
name: {{ env "VAULT_PLUGIN_NAME" | default "quorum" }}
63+
tag: {{ env "VAULT_PLUGIN_TAG" | default "v1.1.3" }}
64+
sha256: {{ env "VAULT_PLUGIN_SHA256SUM" | default "e084800c61749a9c7b51f6e91bb89ab6d5a2678cdb707eaa73f9bef0cf73fc61" }}
65+
filename: {{ env "VAULT_PLUGIN_FILENAME" | default "quorum-hashicorp-vault-plugin" }}
66+
envs:
67+
- VAULT_ADDR: "http://localhost:8200"
4368

4469
kafka:
70+
enabled: true
4571
namespace: {{ env "KAFKA_NAMESPACE" | default "kafka" }}
4672
auth:
4773
enabled: true
@@ -66,9 +92,11 @@ redis:
6692
postgresql:
6793
enabled: true
6894
namespace: {{ env "POSTGRES_NAMESPACE" | default ( env "ORCHESTRATE_NAMESPACE" | default $defaultNamespace ) }}
69-
username: api
70-
password: such-secret
71-
database: api
95+
host: {{ env "POSTGRES_HOST" | default "postgresql" }}
96+
port: 5432
97+
username: {{ env "POSTGRES_USER" | default "api"}}
98+
password: {{ env "POSTGRES_PWD" | default "such-secret" }}
99+
database: {{ env "POSTGRES_DB" | default "api" }}
72100

73101
postgresqlHA:
74102
enabled: false
@@ -79,10 +107,84 @@ postgresqlHA:
79107
database: api
80108
repmgrPassword: such-secret
81109

110+
qkm:
111+
enabled: true
112+
url: "http://quorum-key-manager-quorumkeymanager:8080"
113+
proto: https
114+
fullname: "quorumkeymanager"
115+
orchestrate:
116+
storeName: "eth-accounts"
117+
apiKey: {{ env "QKM_API_KEY" | default "NmwyMCB3YXMgaGVyZQ==" }}
118+
port: 8080
119+
namespace: {{ env "QKM_NAMESPACE" | default $defaultNamespace }}
120+
chart:
121+
name: {{ env "QKM_CHART" | default $defaultQkmChart }}
122+
version: {{ env "QKM_CHART_VERSION" | default $defaultQkmChartVersion }}
123+
124+
image:
125+
repository: {{ env "QKM_REPOSITORY" | default "docker.io/consensys/quorum-key-manager" }}
126+
tag: {{ env "QKM_TAG" | default $qkmTag | quote }}
127+
pullPolicy: Always
128+
129+
b64Manifests: {{ requiredEnv "B64_MANIFESTS" }}
130+
131+
environment: {}
132+
environmentSecrets: {}
133+
134+
postgresql:
135+
tls:
136+
enabled: false
137+
host: {{ env "QKM_POSTGRES_HOST" | default "qkm-postgresql" }}
138+
port: 5432
139+
database: {{ env "QKM_POSTGRES_DB" | default "qkmDB" }}
140+
username: {{ env "QKM_POSTGRES_USER" | default "qkm" }}
141+
password: {{ env "QKM_POSTGRES_PWD" | default "qkmDBPwd" }}
142+
143+
auth:
144+
apikey:
145+
enabled: true
146+
file: "/apikey/api-keys.csv"
147+
oidc:
148+
enabled: false
149+
ca: {{ env "QKM_AUTH_OIDC_CA" }}
150+
pubKey: {{ env "QKM_AUTH_OIDC_PUB_KEY" }}
151+
issuer: {{ env "QKM_AUTH_OIDC_ISSUER_URL" }}
152+
tls:
153+
enabled: false
154+
cacert: {{ env "QKM_AUTH_TLS_CA"}}
155+
secretName: {{ env "QKM_AUTH_TLS_SECRET_NAME" }}
156+
157+
storage:
158+
existingPvc: {{ env "QKM_STORAGE_EXISTING_PVC" | default "" }}
159+
nfs:
160+
enabled: true
161+
driver: {{ env "QKM_STORAGE_NFS_DRIVER" }}
162+
provisioner: {{ env "QKM_STORAGE_NFS_PROVISIONER" }}
163+
fsID: {{ env "QKM_STORAGE_NFS_FS_ID" }}
164+
165+
vault:
166+
tls:
167+
enabled: false
168+
agents: {}
169+
170+
sync:
171+
enabled: false
172+
storeName: {{ env "QKM_SYNC_STORE_NAME" | default "" }}
173+
174+
postgresql-qkm:
175+
enabled: {{ env "DEPLOY_QKM_PG" | default true }}
176+
82177
observability:
83178
namespace: {{ env "OBSERVABILITY_NAMESPACE" | default "observability" }}
84179
grafana:
85180
user: admin
86181
password: frenchfries
87182

88-
domainName: {{ env "DOMAIN_NAME" }}
183+
domainName: {{ env "DOMAIN_NAME" }}
184+
185+
186+
187+
test:
188+
enabled: false
189+
report:
190+
enabled: true

‎environments/default.yaml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,8 @@
11
observability:
2-
enabled: true
2+
enabled: false
33

44
kafka:
5+
enabled: false
56
logRetentionHours: 1
67
resources:
78
requests:
@@ -15,4 +16,3 @@ zookeeper:
1516
memory: "512Mi"
1617
limits:
1718
memory: "1Gi"
18-

‎environments/qa.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ orchestrate:
88
AUTH_API_KEY: with-key
99

1010
observability:
11-
enabled: true
11+
enabled: false
1212

1313
kafka:
1414
numPartitions: 3

0 commit comments

Comments
 (0)