Handle custom ports in upstream URL when no port in proxy URL

[batpad]

I have a situation where my upstream URL is on port 8080 (since stac-auth-proxy will access upstream via an internal kubernetes service URL) - it seems like the logic to over-ride the host does not deal with also over-riding the port: https://github.com/developmentseed/stac-auth-proxy/blob/main/src/stac_auth_proxy/middleware/ProcessLinksMiddleware.py#L96

I think here we also need to check if the links contain a port suffix and remove that (if the stac-auth-proxy hostname does not contain the port)

@alukach does that make sense? does that seem reasonable?

[alukach]

And so you're saying that the links on your API don't contain the correct port?

With regards to the code you linked, we replace based on the URLs netloc property, which contains the port (https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlparse) unlike hostname>

We have a test for this, so this issue would surprise me a bit:

stac-auth-proxy/tests/test_process_links.py

Lines 175 to 217 in 6084e0b

	@pytest.mark.parametrize(
	"upstream_url,root_path,port,input_links,expected_links",
	[
	# Different ports
	(
	"http://upstream.example.com:8080/api",
	"/proxy",
	3000,
	[
	{
	"rel": "self",
	"href": "http://upstream.example.com:8080/api/collections",
	},
	],
	[
	"http://proxy.example.com:3000/proxy/collections",
	],
	),
	],
	)
	def test_transform_upstream_links_with_ports(
	upstream_url, root_path, port, input_links, expected_links
	):
	"""Test transforming upstream links with different ports."""
	middleware = ProcessLinksMiddleware(
	app=None, upstream_url=upstream_url, root_path=root_path
	)
	request_scope = {
	"type": "http",
	"path": "/test",
	"headers": [
	(b"host", f"proxy.example.com:{port}".encode()),
	(b"content-type", b"application/json"),
	],
	}
	data = {"links": input_links}
	transformed = middleware.transform_json(data, Request(request_scope))
	for i, expected in enumerate(expected_links):
	assert transformed["links"][i]["href"] == expected

However, if your concern is that the request isn't making its way upstream, that occurs in our reverse_proxy handler. The URL should be stored as the upstream clients base_url:

stac-auth-proxy/src/stac_auth_proxy/handlers/reverse_proxy.py

Lines 31 to 35 in 6084e0b

	self.client = self.client or httpx.AsyncClient(
	base_url=self.upstream,
	timeout=self.timeout,
	http2=True,
	)

Perhaps you could create a failing test case to demonstrate the issue?

[batpad]

@alukach - interesting indeed - I tried adding a test for my exact case, but I am unable to reproduce the issue in the test.

This is my stac-auth-proxy env config: https://github.com/IFRCGo/go-deploy/blob/develop/applications/argocd/staging/applications/montandon-eoapi.yaml#L124

So I have stac auth proxy at https://stac-test.whydidweevendothis.com and the upstream url is internal on the cluster at http://stac-montandon-eoapi:8080

If I do a curl to https://stac-test.whydidweevendothis.com/collections passing the token in the auth header, in the response I get the href URLs looking like https://stac-test.whydidweevendothis.com:8080/stac/collections/desinventar-events/items instead of https://stac-test.whydidweevendothis.com/collections/desinventar-events/items - so you'll see there's two issues: one is the port in the resultant URL, as well as a /stac/ prefix that im not fully sure where it's coming from.

This is definitely a bit confusing because I was unable to reproduce this behaviour in the tests.

[batpad]

Ok, I think I see roughly what the issue is -

When I try to make a request to http://stac-montandon-eoapi:8080/collections from inside the cluster, the href I get back in the response is like http://stac-montandon-eoapi:8080/stac/collections instead of http://stac-montandon-eoapi:8080/collections - so I think this is something about the setup of the upstream stac-fastapi that thinks there's an extra /stac/ path, and my hunch is because of that, the replace logic in stac-auth-proxy fails and it does not replace the port - but the root cause seems to be in whatever is causing upstream to put the /stac/,

This looks like something I need to dig into upstream and in the eoapi-k8s config, since by default that hosts stac-fastapi at a /stac/ prefix when running all the eoapi tools, but in this case am hitting the stac fastapi service directly so there's no /stac/ prefix in the paths, but some-how stac-fastapi still thinks there is for the URLs in its response. I'll dig, but this seems like not an issue with stac-auth-proxy.

[batpad]

Ok, so we fixed the issue with paths in eoapi-k8s and the issue with ports remains. I was unable to write a test case BUT I was able to reproduce this running stac-auth-proxy and stac-fastapi locally.

Here's a summary of the issue and how to reproduce this locally (you essentially need to run stac-auth-proxy on port 80, and access it with localhost and run upstream on say port 8001. If you query http://localhost/collections then for example, in the URLs in the response will be the upstream port, so URLs will be http://localhost:8001/... )

Here's a concise (AI-generated) summary of the issue and how to reproduce locally:

Issue Description

When accessing stac-auth-proxy via standard ports (80, 443), the URLs in STAC API responses incorrectly include the upstream service port instead of the correct proxy URL.

Expected behavior:
When accessing http://localhost/collections (port 80), response URLs should be:

• "href": "http://localhost/"
• "href": "http://localhost/collections"

Actual behavior:
Response URLs incorrectly show the upstream port:

• "href": "http://localhost:8001/" ❌
• "href": "http://localhost:8001/collections" ❌

This issue only occurs when accessing via standard ports (80, 443). Non-standard ports work correctly.

Steps to Reproduce

1. Start the STAC backend and database:

   docker-compose up -d stac-pg database-pg
   

2. Modify src/stac_auth_proxy/main.py to change port from 8000 to 80:
   port=80, # Change from port=8000

3. Start the proxy on port 80:
   sudo UPSTREAM_URL=http://localhost:8001 OIDC_DISCOVERY_URL=https://goadmin-stage.ifrc.org/o/.well-known/openid-configuration uv run python -m stac_auth_proxy. # you need sudo to run on port 80

4. Test with a valid token:
   curl -H "Authorization: Bearer " http://localhost/collections

Expected vs Actual Results

Expected URLs: http://localhost/ and http://localhost/collections

Actual URLs: http://localhost:8001/ and http://localhost:8001/collections

cc @emmanuelmathot @alukach

[batpad]

Am pretty sure the problem is this line: https://github.com/developmentseed/stac-auth-proxy/blob/main/src/stac_auth_proxy/app.py#L139

So if proxy and upstream are on different ports, it will never call the ProcessLinks middleware?

[batpad]

Yea, if I remove that if statement so the ProcessLinks middleware is always called, then everything works for me as I expect. I'm not fully sure why that if statement exists, etc. - @alukach over to you :)