Helm Chart support

[softcane]

I'm just referring to this thread. #1067 (comment)

->templates
-->cluster-manager-helm-release.yaml(This yaml has the HelmRelease which installs cluster-manager helmchart)
-->rotate-token-cron-job.yaml(This yaml creates a cronjob which runs every 40 minutes generates the required token by running the command:"kubectl create token agent-registration-bootstrap --namespace open-cluster-management" and then generate a kubeconfig and store it somewhere)
-->cluster-manager-post-install.yaml(This yaml has a post install job which creates a initial job from the cronjob since the cronjob only runs every 40 minutes)
->Chart.yaml

custom-ocm-managed-helm(custom helm chart)
->templates
-->klusterlet-helm-release.yaml(This yaml has the HelmRelease which installs klusterlet helmchart)
-->klusterlet-post-install.yaml(This yaml has a post install job which creates the required secret(bootstrap-hub-kubeconfig) after retreiving the kubeconfig generated by hub from storage)
->chart.yaml

When installing the custom-ocm-hub-helm chart, it will generate the kubeconfig and store it in the storage.
While installing the custom-ocm-managed-helm, it will retrieve the secret from storage and create the required secret(bootstrap-hub-kubeconfig).

Where can I find these custom charts? If it's not possible to share the link, then at least can you paste the spec.

[dosubot]

The custom Helm charts you referenced—custom-ocm-hub-helm and custom-ocm-managed-helm—are not part of the official OCM repository or its documentation. They were described as user-created overlays in the thread you linked, but no public repo or YAML specs were shared there either. The official OCM Helm charts for cluster-manager and klusterlet exist and can be found in the repo, but they do not include the automation for token rotation, post-install jobs, or automated secret transfer between clusters as described in your example. Any such automation would need to be implemented by users as custom overlays or additional charts/scripts outside the official OCM charts. You can find the official charts and their usage here:

• cluster-manager Helm chart
• klusterlet Helm chart
  For reference, the discussion about these custom charts and their structure is here: GitHub issue #1067

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[softcane]

@dosu

I'm getting a fowling error. It would be nice to get the spec on how these should be created and updated.

Operation for resource &ResourceAttributes{Namespace:,Verb:create,Group:cluster.open-cluster-management.io,Version:,Resource:managedclusters,Subresource:,Name:,FieldSelector:nil,LabelSelector:nil,} is not allowed with bootstrap secret "open-cluster-management-agent"/"bootstrap-hub-kubeconfig" to apiserver https://xxxx.gr7.us-east-1.eks.amazonaws.com

[qiujian16]

It seems the chart is not maintained by the community. Would like to see the update of the community chart accorindly @jaswalkiranavtar

[softcane]

@qiujian16
I'm looking for some guidance on how to install OCM cluster manager and Klusterlet using helm charts without using the clusteradm command. on eks.

[softcane]

@dosu

I'm getting a fowling error. It would be nice to get the spec on how these should be created and updated.

Operation for resource &ResourceAttributes{Namespace:,Verb:create,Group:cluster.open-cluster-management.io,Version:,Resource:managedclusters,Subresource:,Name:,FieldSelector:nil,LabelSelector:nil,} is not allowed with bootstrap secret "open-cluster-management-agent"/"bootstrap-hub-kubeconfig" to apiserver https://xxxx.gr7.us-east-1.eks.amazonaws.com

[dosubot]

The error means the ServiceAccount/token referenced in your bootstrap-hub-kubeconfig secret doesn't have permission to create ManagedCluster resources on the hub. This is usually an RBAC issue with the bootstrap token or how the secret was created.

Here's how to create and update the bootstrap secret for Helm-based installs (including EKS):

1. On the hub cluster, after installing the cluster-manager Helm chart, get a token for the agent-registration-bootstrap ServiceAccount:

   kubectl -n open-cluster-management create token agent-registration-bootstrap
   

2. Use this token to create a kubeconfig that points to your hub's API server. The user in this kubeconfig should be system:serviceaccount:open-cluster-management:agent-registration-bootstrap.

3. On the managed cluster, create the secret in the open-cluster-management-agent namespace:

   apiVersion: v1
   kind: Secret
   metadata:
     name: bootstrap-hub-kubeconfig
     namespace: open-cluster-management-agent
   type: Opaque
   data:
     kubeconfig: <base64-encoded-kubeconfig>

   Replace <base64-encoded-kubeconfig> with the base64-encoded contents of your kubeconfig file.

4. The ServiceAccount on the hub must be bound to the open-cluster-management:bootstrap ClusterRole, which grants the minimal required permissions:

   • get, list, watch, create, update on CertificateSigningRequest
   • get, list, create, update on ManagedCluster in the cluster.open-cluster-management.io group
     Reference

5. The klusterlet Helm chart will use this secret to register the managed cluster. You can either provide the kubeconfig at install time with --set-file bootstrapHubKubeConfig=<file> or create the secret manually as above after install (klusterlet chart README).

This process is the same for EKS as for other Kubernetes platforms. The key is that the bootstrap token must have the correct RBAC on the hub, and the secret must be present in the right namespace on the managed cluster. If you see this error, double-check the token, the kubeconfig, and the RBAC binding on the hub cluster. For more details, see the official docs.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[softcane]

@dosu

      Failed to create &SelfSubjectAccessReview{ObjectMeta:{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},Spec:SelfSubjectAccessReviewSpec{ResourceAttributes:&ResourceAttributes{Namespace:,Verb:get,Group:certificates.k8s.io,Version:,Resource:certificatesigningrequests,Subresource:,Name:,FieldSelector:nil,LabelSelector:nil,},NonResourceAttributes:nil,},Status:SubjectAccessReviewStatus{Allowed:false,Reason:,EvaluationError:,Denied:false,},} with hub config secret "open-cluster-management-agent"/"hub-kubeconfig-secret" to apiserver https://67EF5CC0239B5F316555CBC2B4DBE708.gr7.us-east-1.eks.amazonaws.com: Post "https://ddd.gr7.us-east-1.eks.amazonaws.com/apis/authorization.k8s.io/v1/selfsubjectaccessreviews": getting credentials: exec: fork/exec /awscli/dist/aws: no such file or directory```

[dosubot]

This error means the kubeconfig in your hub-kubeconfig-secret is using exec-based authentication (like aws eks get-token), but the AWS CLI binary isn't present in the container running the OCM klusterlet controller or agent. This is common on EKS when using IRSA or any kubeconfig that relies on AWS CLI for authentication. The OCM controller tries to run the AWS CLI as specified in the kubeconfig, but fails because the binary isn't there, resulting in the exec: fork/exec /awscli/dist/aws: no such file or directory error.

You have a few options:

• Add the AWS CLI binary to the container image running the klusterlet controller/agent so the exec plugin can run successfully. This is the most direct fix if you want to keep using exec-based kubeconfigs.
• Use a static token-based kubeconfig instead of exec-based authentication if possible (for example, by generating a token on the hub and embedding it directly in the kubeconfig).
• Be aware: this error can cause a false positive "HubConnectionDegraded" status in OCM, but registration and manifestwork sync may still function if the agent pod has the right setup. However, any OCM component or addon that needs to use the kubeconfig will hit the same issue unless the AWS CLI is present and the service account has the right IRSA permissions.

More details and discussion are in this issue and the Helm chart docs. If you need an example Dockerfile snippet to add AWS CLI, let me know.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[softcane]

@dosu

Given this document https://github.com/open-cluster-management-io/ocm/tree/main/solutions/joining-hub-and-spoke-with-aws-auth, where HUB_ROLE_NAME (iam role is created)

[dosubot]

The HUB_ROLE_NAME IAM role referenced in the AWS hub-and-spoke solution needs to be created manually by you, following the instructions and JSON templates in the solution guide. There isn’t an automated script or CloudFormation template in the repo for this—you're expected to use the provided JSON policy/trust templates and replace placeholders with your actual values (like account IDs, cluster names, OIDC provider, etc.) using AWS CLI, CloudFormation, Terraform, or the AWS Console. This setup is required before you can register clusters with IRSA authentication. For details and example templates, see the solution guide and related discussion.

Once the prerequisites are in place, OCM will automatically create a derived IAM role (named like ocm-hub-<md5hash>) during managed cluster acceptance, but the initial HUB_ROLE_NAME and its policies must be set up by you first. If you need help with the actual JSON for the IAM policy or trust relationship, let me know!

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[softcane]

@dosu
'operation error EKS: CreateAccessEntry, https response error StatusCode:
400, RequestID: beac3a7d-ae23-4693-a98d-800be91b25ee, InvalidRequestException:
The cluster''s authentication mode must be set to one of [API, API_AND_CONFIG_MAP]
to perform this operation.'

[dosubot]

This error means your EKS cluster's authentication mode isn't set to allow the CreateAccessEntry operation required by OCM's AWS IRSA integration. EKS clusters must have their authentication mode set to either API or API_AND_CONFIG_MAP for OCM to create access entries and manage IAM roles during registration. This is an AWS cluster-level setting, not managed by OCM itself.

To resolve this, you need to update your EKS cluster's authentication mode using the AWS CLI or AWS Console. The AWS CLI command is:

aws eks update-cluster-config \
  --region <region> \
  --name <cluster-name> \
  --authentication-mode API_AND_CONFIG_MAP

Replace <region> and <cluster-name> with your actual values. You can also set the mode to just API if you don't need ConfigMap-based authentication.

For more details, see the AWS EKS documentation on authentication modes. After updating, retry your OCM operation. This requirement is enforced by AWS and is necessary for OCM's access entry and IAM role automation to work as expected.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}    

[mikeshng]

CC @ramekris3163 in case the AI bot is not giving out the right suggestions.

[softcane]

@ramekris3163 It would be a great help if you could share some guidance and specs to create and rotate these Bootstrap secrets.