K8SPXC-1332 | Allow specifying caBundle for backup storage connection

[mayankshah1607]

CHANGE DESCRIPTION

Problem:

Backups and restores against S3 endpoints served over HTTPS with private/internal certificates fail unless TLS verification is disabled, which weakens security by forcing verifyTLS=false and bypassing server identity checks.

Solution:

Introduce a configurable caBundle for S3 storage so clients can verify TLS using a trusted CA without disabling SSL verification.

Example backup storage

storages:
  minio-s3:
    type: s3
    verifyTLS: true
    s3:
      caBundle:
         name: minio-ca-bundle
         key: tls.crt

CHECKLIST

Jira

• Is the Jira ticket created and referenced properly?
• Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
• Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

• Is an E2E test/test case added for the new feature/change?
• Are unit tests added where appropriate?
• Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

• Are all needed new/changed options added to default YAML files?
• Are all needed new/changed options added to the Helm Chart?
• Did we add proper logging messages for operator actions?
• Did we ensure compatibility with the previous version or cluster upgrade process?
• Does the change support oldest and newest supported PXC version?
• Does the change support oldest and newest supported Kubernetes version?

[egegunes]

ok, let's wait for test results

[mayankshah1607]

@egegunes the test fails in the create_infra function, but I don't see any errors on the logs.. I have no trouble getting it to pass locally. I've copied most of it from the pitr test. Any idea what am I missing? 🤔

[egegunes]

@mayankshah1607 tests are run in cluster wide mode on jenkins. you can run the test with cluster wide by setting OPERATOR_NS=pxc-operator env variable locally.

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

	sed "s/#namespace/$namespace/g" $test_dir/conf/cert.yml |
	kubectl_bin apply -f "-"
	sed "s/#namespace/$namespace/g" $test_dir/conf/cert.yml \
	| kubectl_bin apply -f "-"

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

[github-actions]

[shfmt] _{reported by reviewdog 🐶}

Suggested change

[hors]

@mayankshah1607, please add these new options as a comment in deploy/cr.yaml file

#      caBundle:
#         name: minio-ca-bundle
#         key: tls.crt

@mayankshah1607 Did you create a PR for the Helm chart?

[hors]

I hope it will work for OpenShift

[mayankshah1607]

I haven't tried, but why won't it?

[mayankshah1607]

@hors helm chart PR, please review - percona/percona-helm-charts#688

[JNKPercona]

Test Name	Result	Time
affinity-8-0	passed	00:06:03
auto-tuning-8-0	passed	00:19:02
backup-storage-tls-8-0	passed	00:21:38
cross-site-8-0	passed	00:34:49
custom-users-8-0	failure	00:09:02
demand-backup-cloud-8-0	passed	00:57:25
demand-backup-encrypted-with-tls-8-0	passed	00:44:20
demand-backup-8-0	passed	00:41:40
demand-backup-flow-control-8-0	passed	00:10:36
demand-backup-parallel-8-0	passed	00:08:43
demand-backup-without-passwords-8-0	passed	00:15:08
haproxy-5-7	passed	00:14:41
haproxy-8-0	passed	00:14:48
init-deploy-5-7	passed	00:16:42
init-deploy-8-0	passed	00:16:46
limits-8-0	passed	00:12:05
monitoring-2-0-8-0	passed	00:22:28
monitoring-pmm3-8-0	passed	00:17:35
one-pod-5-7	passed	00:14:28
one-pod-8-0	passed	00:14:07
pitr-8-0	passed	00:42:23
pitr-gap-errors-8-0	passed	00:56:06
proxy-protocol-8-0	passed	00:09:33
proxysql-sidecar-res-limits-8-0	passed	00:08:20
pvc-resize-5-7	passed	00:16:22
pvc-resize-8-0	passed	00:16:32
recreate-8-0	passed	00:17:17
restore-to-encrypted-cluster-8-0	passed	00:25:26
scaling-proxysql-8-0	passed	00:08:35
scaling-8-0	passed	00:10:51
scheduled-backup-5-7	passed	01:04:32
scheduled-backup-8-0	passed	01:03:10
security-context-8-0	passed	00:25:45
smart-update1-8-0	passed	00:33:31
smart-update2-8-0	passed	00:38:32
storage-8-0	passed	00:10:29
tls-issue-cert-manager-ref-8-0	passed	00:08:41
tls-issue-cert-manager-8-0	passed	00:11:24
tls-issue-self-8-0	passed	00:13:04
upgrade-consistency-8-0	passed	00:11:10
upgrade-haproxy-5-7	passed	00:26:28
upgrade-haproxy-8-0	passed	00:25:14
upgrade-proxysql-5-7	passed	00:16:01
upgrade-proxysql-8-0	passed	00:16:28
users-5-7	passed	00:24:33
users-8-0	passed	00:25:43
validation-hook-8-0	passed	00:01:48

Summary	Value
Tests Run	47/47
Job Duration	02:43:26
Total Test Time	17:20:28

commit: f9b155d
image: perconalab/percona-xtradb-cluster-operator:PR-2213-f9b155d0