Skip to content

Conversation

dheeraj-coding
Copy link

@dheeraj-coding dheeraj-coding commented Aug 28, 2025

What this PR does / why we need it:
authentication token is composed of a presigned URL to an STS endpoint. This sts endpoint is simply curled to verify token. There is a possibility that a token from account A with supported regions as HKG, can request authentication against a cluster in account B opted into TLV region.
This behavior breaks data regionalization principle tenets, we must add validations to fail such requests automatically, but this change would be a breaking change to ensure we enforce this behavior in a soft manner we want to track the different number of requests that perform this invalid cross region calls. This PR contains changes necessary to add a new prometheus metric for such cross region calls.

Testing

[root@ip-172-16-190-132 bin]# curl --insecure https://127.0.0.1:21362/metrics | grep sts
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0# HELP aws_iam_authenticator_sts_disabled_region_call Number of STS calls made to regions that are disabled / disabling
# TYPE aws_iam_authenticator_sts_disabled_region_call counter
aws_iam_authenticator_sts_disabled_region_call 0
# HELP aws_iam_authenticator_sts_responses_total Sts responses with error code label
# TYPE aws_iam_authenticator_sts_responses_total counter
aws_iam_authenticator_sts_responses_total{ResponseCode="200",StsRegion="global"} 1
aws_iam_authenticator_sts_responses_total{ResponseCode="200",StsRegion="us-west-2"} 4
100 11632    0 11632    0     0  1187k      0 --:--:-- --:--:-- --:--:-- 1262k
# HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
# TYPE promhttp_metric_handler_requests_in_flight gauge
promhttp_metric_handler_requests_in_flight 1
# HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
# TYPE promhttp_metric_handler_requests_total counter
promhttp_metric_handler_requests_total{code="200"} 67
promhttp_metric_handler_requests_total{code="500"} 0
promhttp_metric_handler_requests_total{code="503"} 0

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: dheeraj-coding
Once this PR has been reviewed and has the lgtm label, please assign micahhausler for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Aug 28, 2025
@k8s-ci-robot
Copy link
Contributor

Welcome @dheeraj-coding!

It looks like this is your first PR to kubernetes-sigs/aws-iam-authenticator 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/aws-iam-authenticator has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

@k8s-ci-robot k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Aug 28, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @dheeraj-coding. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Aug 28, 2025
instanceRegion = instanceRegionOutput.Region
}

acctClient := account.NewFromConfig(cfg)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would need additional permissions which could be breaking change. Is there a need for this, can we get any information based on failure code from sts?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Today, such cross region calls do not fail, but they succeed as the bearer token that we validate is simply a presignedURL. Just tried reproducing this behavior locally,

  1. Use account A with region HKG
$ aws-iam-authenticator token -i 126db5ea-e932-4251-b738-818fee4ed20c --role arn:aws:iam::509399613853:role/Admin --token-only --region ap-east-1
  1. Use the generated token and validate the with credentials exported for account B
$ aws-iam-authenticator verify -t <token> -i 126db5ea-e932-4251-b738-818fee4ed20c
Response:
&{ARN:arn:aws:sts::509399613853:assumed-role/Admin/aws-go-sdk-1756362745475065571 CanonicalARN:arn:aws:iam::509399613853:role/Admin AccountID:509399613853 UserID:AROAXNGUVIGO4AYZOL27A SessionName:aws-go-sdk-1756362745475065571 AccessKeyID:ASIAXNGUVIGOZP6UQQZD}

There is no error from STS to enforce this behavior. I could make it fail silently just like when imds clients are unavailable ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

iam roles are global, is there a reason we think token from one region should not be allowed in other region? We have separation at the partition level currently.

@k8s-ci-robot k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 4, 2025
@k8s-ci-robot
Copy link
Contributor

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants