Implement ingress for easier services exposing #1314
Conversation
|
This is a prerequisite for https://github.com/e2b-dev/belt/pull/217 that shows exposing the service via an ingress service. |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| } | ||
| } | ||
|
|
||
| resource "google_compute_url_map" "ingress" { |
There was a problem hiding this comment.
Can we get a better name here? I think we now have two load balancers, "ingress" and "orch_map", neither of their names help understand what they do. Maybe "traefik" and "direct"?
There was a problem hiding this comment.
I like ingress as its common name and it describes what it is. Yes, "orch_map" is, in my opinion, a mistake, as it no longer makes sense. Ideally, I would like to transition away from the current load balancer once the migration is complete/rename it to something like "ingress-sandboxes" or a similar name to distinguish better.
I don't like to call it Traefik, as we can switch the ingress backend at any time in the future, but I'm okay with you coming up with a better name.
There was a problem hiding this comment.
a note here, if we want to rename the orch_map to ingress-sandboxes, maybe we should name this ingress something like ingress-api or ingress-management or ingress-services
There was a problem hiding this comment.
I still don’t like that we would need two load balancers just because we cannot filter sandbox traffic. Will look into again tomorrow
Yep, we can rename ingress to something else. Iam not sure about management/api as we can use it for something different in future. Ingress services sounds okay to me.
There was a problem hiding this comment.
I though it's actually quite nice to have separate LBs for user's sandbox traffic and our services traffic (different limitations, limits, HTTP support, etc), but maybe it's unnecessary
There was a problem hiding this comment.
Ideally, we should be able to match sandbox traffic to different rules (now it's catch-all fallback) so we can apply different limits/armor rules to them, then we don't need to have different LBs.
For supporting newer versions of HTTP, etc, we can still relatively easily migrate everything, and I'm not sure if we would need some special LB that cannot handle both sandbox and services traffic.
| cpu_count = 1 | ||
| memory_mb = 512 |
There was a problem hiding this comment.
are these enough for our traffic?
There was a problem hiding this comment.
Yes, more than enough for now. When we proceed and start routing sandbox traffic, we need to revisit this, with memory ideally set to 1024 min and 2048 max.
| $(call tfvar, GCP_REGION) \ | ||
| $(call tfvar, GCP_ZONE) \ | ||
| $(call tfvar, DOMAIN_NAME) \ | ||
| $(call tfvar, ADDITIONAL_DOMAINS) \ |
There was a problem hiding this comment.
note here, this is a breaking change and should be handled with proper migration path
There was a problem hiding this comment.
Do we have any? This type transformation @jakubno we handles manually.
There was a problem hiding this comment.
Maybe I can push attribute before this pr
There was a problem hiding this comment.
can we maybe do the secret default value from the env var? Is that a bad idea?
There was a problem hiding this comment.
Imho during migration period we can merge server + tf var lists.
There was a problem hiding this comment.
Added compatibility during the migration period that will accept both env and secret and merge them together.
| "--providers.consulcatalog=true", | ||
| "--providers.consulcatalog.exposedByDefault=false", | ||
| "--providers.consulcatalog.endpoint.address=${consul_endpoint}", | ||
| "--providers.consulcatalog.endpoint.token=${consul_token}", |
There was a problem hiding this comment.
Bug: Token Exposure and Network Resolution Issues
Sensitive Nomad and Consul tokens are passed as command-line arguments to the Traefik container, exposing them in process lists and logs. Additionally, if the nomad_endpoint is configured as localhost, it may not resolve correctly from within the container using host networking, potentially causing service discovery failures.
Note
Deploys a Traefik-based ingress via Nomad and GCP HTTPS load balancer, and moves additional routing domains to Secret Manager with module-wide wiring.
ingressrunningtraefik:v3.5with Nomad/Consul providers; newingress_countandingress_portvariables.api_instance_group, URL map, target HTTPS proxy, global IP/forwarding rule; reference existing certificate map.ingress_portacross modules (variables.tf,main.tf), add API instance group named port, and open firewall for the ingress port.routing-domainsand initial version; outputrouting_domains_secret_name.additional_domainsin root module and pass downstream.INGRESS_COUNTTF var.Written by Cursor Bugbot for commit 9e24600. This will update automatically on new commits. Configure here.