RBAC Permissions Updates to Service Accounts on `dev-vegbank` and `dev-vegbank-dev`

[doulikecookiedough]

Hi @artntek,

May I please get your assistance to update the service account permissions to manage CNPG resources when you return from your vacation for both service accounts on dev-vegbank and dev-vegbank-dev contexts? I also need the patch verb for cnpg.

• Context Note: The existing SA on dev-vegbank still does not have sufficient permissions - it did not successfully update with my previous mishap.

Context:

NCEAS/vegbank2#215

I am trying to perform a minor upgrade on my existing postgres:17.5 cnpg cluster for vegbankdb to postgres:17.6 - as this is the recommended version for the cnpg cluster to be running before performing an in-place upgrade via pg_upgrade. I feel that this path is the most straightforward with the least cognitive load, and we would follow these steps:

• Stop the vegbank API from receiving requests
• Create an immediate snapshot/backup with ScheduledBackup via kubectl patch scheduledbackup name-of-backup ...
• Recover from the ScheduledBackup with a new release name (ex. vegbankdb18)
• Perform the in-place upgrade via pg_upgrade
• Upgrade the vegbank API to point to the new cnpg cluster
• Test uploads and API requests
• If everything looks good, create a full pg_dump file as a safety/precautionary measure
• Uninstall the helm chart for the old cluster

Reference code/configuration snippets:

How to check existing cnpg cluster version:

$ kubectl cnpg psql vegbankdb-cnpg -- -qAt -c 'SELECT version()' 
PostgreSQL 17.5 (Debian 17.5-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit

Update to my values-cnpg.yaml to change versions

# values-cnpg.yaml

## @section CNPG spec - Specifications for the CNPG cluster
spec:
  ## @param spec.imageName Adjust this value to your desired postgres version
  ## To see available versions, see: https://github.com/cloudnative-pg/postgres-containers/pkgs/container/postgresql
  imageName: ghcr.io/cloudnative-pg/postgresql:17.6

Failure to upgrade due to insufficient permissions:

$ helm upgrade vegbankdb18 oci://ghcr.io/dataoneorg/charts/cnpg -f '/Users/doumok/Code/vegbank2/helm/values-cnpg.yaml' --debug
DEBU[0000] resolving                                     host=ghcr.io
DEBU[0000] do request                                    host=ghcr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=Helm/3.15.3 request.method=HEAD url="https://ghcr.io/v2/dataoneorg/charts/cnpg/manifests/1.0.0"
DEBU[0000] fetch response received                       host=ghcr.io response.header.content-length=73 response.header.content-type=application/json response.header.date="Mon, 29 Dec 2025 20:04:20 GMT" response.header.www-authenticate="Bearer realm=\"https://ghcr.io/token\",service=\"ghcr.io\",scope=\"repository:dataoneorg/charts/cnpg:pull\"" response.header.x-github-request-id="FF08:18BC51:CC60F:F3B56:6952DEC4" response.status="401 Unauthorized" url="https://ghcr.io/v2/dataoneorg/charts/cnpg/manifests/1.0.0"
DEBU[0000] Unauthorized                                  header="Bearer realm=\"https://ghcr.io/token\",service=\"ghcr.io\",scope=\"repository:dataoneorg/charts/cnpg:pull\"" host=ghcr.io
DEBU[0001] do request                                    host=ghcr.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=Helm/3.15.3 request.method=HEAD url="https://ghcr.io/v2/dataoneorg/charts/cnpg/manifests/1.0.0"
DEBU[0001] fetch response received                       host=ghcr.io response.header.content-length=707 response.header.content-type=application/vnd.oci.image.manifest.v1+json response.header.date="Mon, 29 Dec 2025 20:04:21 GMT" response.header.docker-content-digest="sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27\"" response.header.strict-transport-security="max-age=63072000; includeSubDomains; preload" response.header.x-github-request-id="FF08:18BC51:CC660:F3BBE:6952DEC5" response.status="200 OK" url="https://ghcr.io/v2/dataoneorg/charts/cnpg/manifests/1.0.0"
DEBU[0001] resolved                                      desc.digest="sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27" host=ghcr.io
DEBU[0001] do request                                    digest="sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27" request.header.accept="application/vnd.oci.image.manifest.v1+json, */*" request.header.user-agent=Helm/3.15.3 request.method=GET url="https://ghcr.io/v2/dataoneorg/charts/cnpg/manifests/sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27"
DEBU[0001] fetch response received                       digest="sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27" response.header.content-length=707 response.header.content-type=application/vnd.oci.image.manifest.v1+json response.header.date="Mon, 29 Dec 2025 20:04:21 GMT" response.header.docker-content-digest="sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27\"" response.header.strict-transport-security="max-age=63072000; includeSubDomains; preload" response.header.x-github-request-id="FF08:18BC51:CC685:F3BE6:6952DEC5" response.status="200 OK" url="https://ghcr.io/v2/dataoneorg/charts/cnpg/manifests/sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27"
DEBU[0001] do request                                    digest="sha256:fc7182128d3275b19271bdf202dc9355631aad9ef225bd1caea43557a19c714f" request.header.accept="application/vnd.cncf.helm.chart.content.v1.tar+gzip, */*" request.header.user-agent=Helm/3.15.3 request.method=GET url="https://ghcr.io/v2/dataoneorg/charts/cnpg/blobs/sha256:fc7182128d3275b19271bdf202dc9355631aad9ef225bd1caea43557a19c714f"
DEBU[0001] do request                                    digest="sha256:2b0443b821657f8c345b693fe2ee69197f4ba05aa49e2762bb824ebfcb9470cf" request.header.accept="application/vnd.cncf.helm.config.v1+json, */*" request.header.user-agent=Helm/3.15.3 request.method=GET url="https://ghcr.io/v2/dataoneorg/charts/cnpg/blobs/sha256:2b0443b821657f8c345b693fe2ee69197f4ba05aa49e2762bb824ebfcb9470cf"
DEBU[0001] fetch response received                       digest="sha256:2b0443b821657f8c345b693fe2ee69197f4ba05aa49e2762bb824ebfcb9470cf" response.header.accept-ranges=bytes response.header.age=65 response.header.content-disposition= response.header.content-length=216 response.header.content-type=application/octet-stream response.header.date="Mon, 29 Dec 2025 20:04:21 GMT" response.header.etag="\"0x8DE2D475807AD91\"" response.header.last-modified="Wed, 26 Nov 2025 23:55:57 GMT" response.header.server="Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0" response.header.strict-transport-security="max-age=31536000" response.header.via="1.1 varnish, 1.1 varnish" response.header.x-cache="MISS, HIT" response.header.x-cache-hits="0, 0" response.header.x-fastly-request-id=5209401b485fb9e2e931b880030aa4cbf56dfe6c response.header.x-ms-blob-type=BlockBlob response.header.x-ms-copy-completion-time="Wed, 26 Nov 2025 23:55:57 GMT" response.header.x-ms-copy-id=248ec1e0-d358-4ed0-94f0-bdeb8b5318fa response.header.x-ms-copy-progress=216/216 response.header.x-ms-copy-status=success response.header.x-ms-creation-time="Wed, 26 Nov 2025 23:55:57 GMT" response.header.x-ms-last-access-time="Mon, 29 Dec 2025 18:55:13 GMT" response.header.x-ms-lease-state=available response.header.x-ms-lease-status=unlocked response.header.x-ms-request-id=f0b8ff69-701e-0074-2cfe-78ada4000000 response.header.x-ms-server-encrypted=true response.header.x-ms-version=2025-01-05 response.header.x-served-by="cache-iad-kiad7000105-IAD, cache-bur-kbur8200104-BUR" response.status="200 OK" url="https://ghcr.io/v2/dataoneorg/charts/cnpg/blobs/sha256:2b0443b821657f8c345b693fe2ee69197f4ba05aa49e2762bb824ebfcb9470cf"
DEBU[0001] fetch response received                       digest="sha256:fc7182128d3275b19271bdf202dc9355631aad9ef225bd1caea43557a19c714f" response.header.accept-ranges=bytes response.header.age=65 response.header.content-disposition= response.header.content-length=21640 response.header.content-type=application/octet-stream response.header.date="Mon, 29 Dec 2025 20:04:21 GMT" response.header.etag="\"0x8DE311B88B4BA05\"" response.header.last-modified="Mon, 01 Dec 2025 20:52:26 GMT" response.header.server="Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0" response.header.strict-transport-security="max-age=31536000" response.header.via="1.1 varnish, 1.1 varnish" response.header.x-cache="MISS, HIT" response.header.x-cache-hits="0, 0" response.header.x-fastly-request-id=8d4980c50cbb16abdc736330f2e0ff65bfadeced response.header.x-ms-blob-type=BlockBlob response.header.x-ms-copy-completion-time="Mon, 01 Dec 2025 20:52:26 GMT" response.header.x-ms-copy-id=40692f86-5645-46eb-88b9-1857547c749b response.header.x-ms-copy-progress=21640/21640 response.header.x-ms-copy-status=success response.header.x-ms-creation-time="Mon, 01 Dec 2025 20:52:26 GMT" response.header.x-ms-last-access-time="Mon, 29 Dec 2025 18:55:13 GMT" response.header.x-ms-lease-state=available response.header.x-ms-lease-status=unlocked response.header.x-ms-request-id=1671dd6d-c01e-0101-1afe-781a1c000000 response.header.x-ms-server-encrypted=true response.header.x-ms-version=2025-01-05 response.header.x-served-by="cache-iad-kiad7000071-IAD, cache-bur-kbur8200104-BUR" response.status="200 OK" url="https://ghcr.io/v2/dataoneorg/charts/cnpg/blobs/sha256:fc7182128d3275b19271bdf202dc9355631aad9ef225bd1caea43557a19c714f"
DEBU[0001] encountered unknown type application/vnd.cncf.helm.config.v1+json; children may not be fetched 
DEBU[0001] encountered unknown type application/vnd.cncf.helm.chart.content.v1.tar+gzip; children may not be fetched 
Pulled: ghcr.io/dataoneorg/charts/cnpg:1.0.0
Digest: sha256:b9b257d51280309b12720285d7dbfbbf631c88f2dd7af34a2fa9f4c372e97a27
upgrade.go:158: [debug] preparing upgrade for vegbankdb18
upgrade.go:166: [debug] performing update for vegbankdb18
upgrade.go:364: [debug] creating upgraded release for vegbankdb18
client.go:393: [debug] checking 2 resources for changes
client.go:693: [debug] Patch Cluster "vegbankdb18-cnpg" in namespace vegbank-dev
client.go:693: [debug] Patch ScheduledBackup "vegbankdb18-scheduled-backup" in namespace vegbank-dev
client.go:425: [debug] error updating the resource "vegbankdb18-scheduled-backup":
         cannot patch "vegbankdb18-scheduled-backup" with kind ScheduledBackup: scheduledbackups.postgresql.cnpg.io "vegbankdb18-scheduled-backup" is forbidden: User "system:serviceaccount:vegbank-dev:vegbank-dev" cannot patch resource "scheduledbackups" in API group "postgresql.cnpg.io" in the namespace "vegbank-dev"
upgrade.go:476: [debug] warning: Upgrade "vegbankdb18" failed: cannot patch "vegbankdb18-scheduled-backup" with kind ScheduledBackup: scheduledbackups.postgresql.cnpg.io "vegbankdb18-scheduled-backup" is forbidden: User "system:serviceaccount:vegbank-dev:vegbank-dev" cannot patch resource "scheduledbackups" in API group "postgresql.cnpg.io" in the namespace "vegbank-dev"
Error: UPGRADE FAILED: cannot patch "vegbankdb18-scheduled-backup" with kind ScheduledBackup: scheduledbackups.postgresql.cnpg.io "vegbankdb18-scheduled-backup" is forbidden: User "system:serviceaccount:vegbank-dev:vegbank-dev" cannot patch resource "scheduledbackups" in API group "postgresql.cnpg.io" in the namespace "vegbank-dev"
helm.go:84: [debug] cannot patch "vegbankdb18-scheduled-backup" with kind ScheduledBackup: scheduledbackups.postgresql.cnpg.io "vegbankdb18-scheduled-backup" is forbidden: User "system:serviceaccount:vegbank-dev:vegbank-dev" cannot patch resource "scheduledbackups" in API group "postgresql.cnpg.io" in the namespace "vegbank-dev"
helm.sh/helm/v3/pkg/kube.(*Client).Update
        helm.sh/helm/v3/pkg/kube/client.go:438
helm.sh/helm/v3/pkg/action.(*Upgrade).releasingUpgrade
        helm.sh/helm/v3/pkg/action/upgrade.go:418
runtime.goexit
        runtime/asm_arm64.s:1222
UPGRADE FAILED
main.newUpgradeCmd.func2
        helm.sh/helm/v3/cmd/helm/upgrade.go:240
github.com/spf13/cobra.(*Command).execute
        github.com/spf13/cobra@v1.8.0/command.go:983
github.com/spf13/cobra.(*Command).ExecuteC
        github.com/spf13/cobra@v1.8.0/command.go:1115
github.com/spf13/cobra.(*Command).Execute
        github.com/spf13/cobra@v1.8.0/command.go:1039
main.main
        helm.sh/helm/v3/cmd/helm/helm.go:83
runtime.main
        runtime/proc.go:271
runtime.goexit
        runtime/asm_arm64.s:1222