fix: performance recession when propagating namespace to helm #5971

Skaronator · 2025-08-25T07:27:52Z

This PR addresses the significant performance regression introduced by #5940 (namespace propagation to Helm charts). The original implementation
required expensive resource origin tracking, causing build times to increase from ~0.6s to over 13s.

Solution

Instead of using expensive resource origin tracking, this PR introduces a lightweight annotation-based approach:

Helm chart resources are now tagged with kustomize.config.k8s.io/helm-chart-generated: "true" annotation during generation
NamespaceTransformer uses this annotation to efficiently identify and skip Helm-generated resources
Eliminates the performance bottleneck of origin tracking while maintaining the same functionality

Performance Impact

Version	Build Time	Improvement
Latest master	15.34s	baseline
Latest release	0.59s	-
This PR	0.54s	96.5% faster than master

Technical Notes

The annotation approach is ~30x faster than origin tracking
Resource tracking performance issues remain a broader architectural concern for future consideration

k8s-ci-robot · 2025-08-25T07:28:02Z

Hi @Skaronator. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-08-25T09:01:52Z

This PR has multiple commits, and the default merge method is: merge.
You can request commits to be squashed using the label: tide/merge-method-squash

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

ephesused · 2025-08-25T13:29:27Z

Thanks, @Skaronator. Since you're already working on this one, would you be willing to include the test from #2869? If not, I can plan to do it myself, but that's likely to be a little while - I won't have the time for at least a few days.

Skaronator · 2025-08-25T13:38:49Z

Hi @ephesused,
The test from #2869 seems independent from this fix and would be better as its own PR - that way we don't block this one from merging.
I'm quite busy right now but can look into adding that benchmark, maybe next week if you haven't already done it yourself.

ephesused · 2025-08-25T14:18:01Z

@Skaronator, no worries. Thanks for this fix!

ephesused · 2025-08-25T15:01:12Z

I ran this change through the same testing process I noted over in #5940 (comment). This change looks good to me:

kustomize_test.go:293: Runtime variation (15.56x): branch-1c0f1bf5 ran in 9619ms; release-v5.4.3 ran in 618ms.
...
kustomize_test.go:293: Runtime variation (0.96x): branch-pr-5971 ran in 596ms; release-v5.4.3 ran in 618ms.

This reverts commit 976a7cf.

This reverts commit c7612d1.

Skaronator · 2025-08-26T14:07:12Z

Reverted the origin tracking which causes the performance issue. Now relying on a annotation to keep track that these resources are generated by Helm.

Long term it would be probably better to revisit the Origin Tracking feature. That feature is behind a feature flag but the 20x slowdown still exist.

This reverts commit 8948788.

Skaronator · 2025-09-06T14:30:32Z

Tests now pass. Somehow forget to commit that one file.

plugin/builtin/namespacetransformer/NamespaceTransformer_test.go

koba1t · 2025-09-12T23:32:18Z

/ok-to-test

stormqueen1990 · 2025-09-14T21:04:11Z

/uncc

Skaronator · 2025-09-17T21:37:17Z

/retest

Skaronator · 2025-09-17T21:38:54Z

/test kustomize-presubmit-master

Skaronator · 2025-09-17T22:31:29Z

kustomize-presubmit-master now passes. Can anyone trigger the GitHub Actions again? Not sure why they don't run. I'm already a Contributor and the PR has ok-to-test label.

api/resource/resource.go

api/internal/target/kusttarget.go

ephesused · 2025-09-23T20:10:27Z

Still LGTM:

    kustomize_test.go:296: Runtime variation (21.08x): branch-master ran in 13722ms; release-v5.4.3 ran in 651ms.
    kustomize_test.go:298: Marking test a failure due to runtime variation

...

            --- PASS: Test_kustomize/kust-issue-2869/kustomize/release-v5.7.1 (0.69s)
            --- FAIL: Test_kustomize/kust-issue-2869/kustomize/branch-master (13.74s)
            --- PASS: Test_kustomize/kust-issue-2869/kustomize/branch-pr-5971 (0.58s)

koba1t

I think looks like almost good with code review.
Please fix this comments.

plugin/builtin/helmchartinflationgenerator/HelmChartInflationGenerator.go

koba1t · 2025-09-29T19:14:07Z

@Skaronator

Resource tracking performance issues remain a broader architectural concern for future consideration

Could you open new issue that tracking this performance problem?

Skaronator · 2025-09-29T19:32:35Z

@koba1t

Could you open new issue that tracking this performance problem?

Yeah that seems like a good idea. I created #5988.

koba1t · 2025-09-29T21:07:39Z

Thanks @Skaronator
I think this change is almost good!

And thanks for your help @ephesused!

/lgtm
/approve

k8s-ci-robot · 2025-09-29T21:07:47Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: koba1t, Skaronator

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [koba1t]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [kubernetes-sigs/kustomize](https://github.com/kubernetes-sigs/kustomize) | minor | `v5.7.1` -> `v5.8.0` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>kubernetes-sigs/kustomize (kubernetes-sigs/kustomize)</summary> ### [`v5.8.0`](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize/v5.8.0) [Compare Source](kubernetes-sigs/kustomize@kustomize/v5.7.1...kustomize/v5.8.0) ### Highlights ##### implements to replacements value in the structured data Now, We can edit yaml/json in yaml manifests with replacements transformer. See [#5679](kubernetes-sigs/kustomize#5679) ##### For example ```yaml ## source apiVersion: v1 kind: ConfigMap metadata: name: source-configmap data: HOSTNAME: www.example.com --- apiVersion: v1 kind: ConfigMap metadata: name: target-configmap data: config.json: |- {"config": { "id": "42", "hostname": "REPLACE_TARGET_HOSTNAME" }} ``` ```yaml ## replacement replacements: - source: kind: ConfigMap name: source-configmap fieldPath: data.HOSTNAME targets: - select: kind: ConfigMap name: target-configmap fieldPaths: - data.config\.json.config.hostname ``` ##### fix: Propagate Namespace correctly to Helm The long-standing bug where kustomize's namespace transformer did not pass namespaces to helmCharts has been fixed. See [#5940](kubernetes-sigs/kustomize#5940) ##### For example ```yaml ## define namespace namespace: any-namespace helmCharts: - name: minecraft repo: https://kubernetes-charts.storage.googleapis.com version: v1.2.0 # namespace: any-namespace ## propagates without additional namespace specific valuesFile: values.yaml ``` #### Feature [#5679](kubernetes-sigs/kustomize#5679): implements to replacements value in the structured data [#5863](kubernetes-sigs/kustomize#5863): Add regex support for Replacement selectors [#5930](kubernetes-sigs/kustomize#5930): feat: add PatchArgs API type to populate patch options #### fix [#5940](kubernetes-sigs/kustomize#5940): fix: Propagate Namespace correctly to Helm [#5971](kubernetes-sigs/kustomize#5971): fix: performance recession when propagating namespace to helm [#5942](kubernetes-sigs/kustomize#5942): fix fnplugin storagemounts validation [#5958](kubernetes-sigs/kustomize#5958): fix: make AbsorbAll conflict error more verbose [#5961](kubernetes-sigs/kustomize#5961): refactor: nested format string [#5967](kubernetes-sigs/kustomize#5967): Fix infinite loop in HTTP client by validating URLs before requests [#5985](kubernetes-sigs/kustomize#5985): fix(kyaml/yaml): minor nil safety fix for RNode.Content etc [#5991](kubernetes-sigs/kustomize#5991): Fix duplicate key error when adding multiple labels with --without-selector #### Dependencies [#5962](kubernetes-sigs/kustomize#5962): chore: update dependencies from security alert [#5959](kubernetes-sigs/kustomize#5959): update go 1.24.6 #### chore [#6007](kubernetes-sigs/kustomize#6007): Update kyaml to v0.21.0 [#6008](kubernetes-sigs/kustomize#6008): Update cmd/config to v0.21.0 [#6009](kubernetes-sigs/kustomize#6009): Update api to v0.21.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 25, 2025

k8s-ci-robot requested review from stormqueen1990 and varshaprasad96 August 25, 2025 07:28

k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 25, 2025

fix: performance recession when propagating namespace to helm

c7612d1

Skaronator force-pushed the performance-improvments branch from 20486a7 to c7612d1 Compare August 25, 2025 07:36

fix: handle passing namespace downstream more elegant

976a7cf

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Aug 25, 2025

Skaronator marked this pull request as draft August 25, 2025 21:22

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 25, 2025

Skaronator added 4 commits August 26, 2025 12:08

Revert "fix: handle passing namespace downstream more elegant"

97d3a7c

This reverts commit 976a7cf.

Revert "fix: performance recession when propagating namespace to helm"

c8186c7

This reverts commit c7612d1.

fix: use annotation to identify helm chart generated resources

c77d174

fix: deduplicate code

7c9eec4

Skaronator marked this pull request as ready for review August 26, 2025 14:05

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 26, 2025

k8s-ci-robot requested a review from koba1t August 26, 2025 14:06

Skaronator added 3 commits September 6, 2025 15:58

fix: missing import in NamespaceTransformer.go

246a39d

ci: allow manual trigger of pipeline in fork

8948788

Revert "ci: allow manual trigger of pipeline in fork"

9a5469c

This reverts commit 8948788.

koba1t reviewed Sep 12, 2025

View reviewed changes

plugin/builtin/namespacetransformer/NamespaceTransformer_test.go Outdated Show resolved Hide resolved

koba1t added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Sep 12, 2025

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Sep 12, 2025

k8s-ci-robot removed the request for review from stormqueen1990 September 14, 2025 21:04

fix: test cases

9ee5994

Skaronator commented Sep 17, 2025

View reviewed changes

api/resource/resource.go Show resolved Hide resolved

Skaronator commented Sep 17, 2025

View reviewed changes

api/internal/target/kusttarget.go Show resolved Hide resolved

Skaronator requested a review from koba1t September 21, 2025 19:36

koba1t requested changes Sep 29, 2025

View reviewed changes

plugin/builtin/helmchartinflationgenerator/HelmChartInflationGenerator.go Outdated Show resolved Hide resolved

Skaronator added 2 commits September 29, 2025 21:20

chore: fix code comment was on wrong line

f91fd87

chore: fix code comment was on wrong line pt2

a1f403b

Skaronator mentioned this pull request Sep 29, 2025

Severe performance degradation when enabling resource tracking #5988

Open

k8s-ci-robot assigned koba1t Sep 29, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Sep 29, 2025

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Sep 29, 2025

k8s-ci-robot merged commit 278dd6e into kubernetes-sigs:master Sep 29, 2025
11 checks passed

fix: performance recession when propagating namespace to helm #5971

fix: performance recession when propagating namespace to helm #5971

Uh oh!

Conversation

Skaronator commented Aug 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Solution

Performance Impact

Technical Notes

Uh oh!

k8s-ci-robot commented Aug 25, 2025

Uh oh!

k8s-ci-robot commented Aug 25, 2025

Uh oh!

ephesused commented Aug 25, 2025

Uh oh!

Skaronator commented Aug 25, 2025

Uh oh!

ephesused commented Aug 25, 2025

Uh oh!

ephesused commented Aug 25, 2025

Uh oh!

Skaronator commented Aug 26, 2025

Uh oh!

Skaronator commented Sep 6, 2025

Uh oh!

Uh oh!

koba1t commented Sep 12, 2025

Uh oh!

stormqueen1990 commented Sep 14, 2025

Uh oh!

Skaronator commented Sep 17, 2025

Uh oh!

Skaronator commented Sep 17, 2025

Uh oh!

Skaronator commented Sep 17, 2025

Uh oh!

Uh oh!

Uh oh!

ephesused commented Sep 23, 2025

Uh oh!

koba1t left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

koba1t commented Sep 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Skaronator commented Sep 29, 2025

Uh oh!

koba1t commented Sep 29, 2025

Uh oh!

k8s-ci-robot commented Sep 29, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Skaronator commented Aug 25, 2025 •

edited

Loading

koba1t commented Sep 29, 2025 •

edited

Loading