Skip to content

Adding Kubvernor to the list of implementors #1313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 13 commits into from
Oct 4, 2025
Merged

Adding Kubvernor to the list of implementors #1313

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
96f2ceb
Adding Kubvernor section to the implementors list
dawid-nowak Aug 6, 2025
db14992
Merge branch 'main' into adding_kubvernor_implementor_v2
dawid-nowak Aug 13, 2025
426f4c0
Addressing PR comments
dawid-nowak Aug 25, 2025
8dcb895
Merge remote-tracking branch 'origin' into adding_kubvernor_implement…
dawid-nowak Aug 25, 2025
8e66626
Addressing PR comments
dawid-nowak Aug 25, 2025
061a556
Update site-src/implementations/gateways.md
dawid-nowak Aug 26, 2025
5719437
PR Review
dawid-nowak Sep 3, 2025
f3895cf
Merge remote-tracking branch 'refs/remotes/origin/adding_kubvernor_im…
dawid-nowak Sep 3, 2025
4e2ac44
PR Review
dawid-nowak Sep 3, 2025
2847818
Removing extra file
dawid-nowak Sep 4, 2025
8b85635
Merge master and for time being remove Kubvernor from the guides
dawid-nowak Sep 21, 2025
6eec771
Removing conformance mention from the description
dawid-nowak Sep 25, 2025
4683dca
Merge remote-tracking branch 'origin/main' into adding_kubvernor_impl…
dawid-nowak Sep 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions config/manifests/gateway/kubvernor/gateway.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
name: inference-gateway
spec:
gatewayClassName: kubvernor-inference-gateway
listeners:
- name: http
port: 80
protocol: HTTP
20 changes: 20 additions & 0 deletions config/manifests/gateway/kubvernor/httproute.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
name: llm-route
spec:
parentRefs:
- group: gateway.networking.k8s.io
kind: Gateway
name: inference-gateway
rules:
- backendRefs:
- group: inference.networking.x-k8s.io
kind: InferencePool
name: vllm-llama3-8b-instruct
matches:
- path:
type: PathPrefix
value: /
timeouts:
request: 300s
39 changes: 39 additions & 0 deletions site-src/guides/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,42 @@ A cluster with:
kubectl get httproute llm-route -o yaml
```

=== "Kubvernor Rust API Gateway"

[Kubvernor Rust API Gateway](https://github.com/kubvernor/kubvernor) is a higly experimental project so not ready for production but it supports version v0.5.1 of Inference Extension Spec.

1. Requirements
- Rust and Cargo installed
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A user should not be required to install Rust to deploy a GIE gateway. Common deployment tools such as kubectl, Helm, etc., should be the only requirement for a user to deploy.


2. Run Kubvernor Rust API Gateway as documented in [README](https://github.com/kubvernor/kubvernor/blob/main/README.md)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All required steps should be captured here instead of an external link

Note: Due to #1502, we plan on removing most 3rd party content from the docs site. We will instead refer to the implementations page. Conformant implementations will be required to maintain their own install guides.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the referenced readme but the ./run_kubvernor.sh step failed:

$ rustc --version
rustc 1.89.0 (29483883e 2025-08-04)
$ cargo version
cargo 1.89.0 (c24e10642 2025-06-23)
$ echo $CONTROL_PLANE_IP
172.18.0.2
$ ./run_kubvernor.sh
...
  --- stderr
  Error: Custom { kind: Other, error: "protoc failed: google/protobuf/timestamp.proto: File not found.\ngoogle/protobuf/descriptor.proto: File not found.\nudpa/annotations/status.proto: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\nudpa/annotations/status.proto:15:8: \"google.protobuf.FileOptions\" is not defined.\nudpa/annotations/versioning.proto: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\nudpa/annotations/versioning.proto:13:8: \"google.protobuf.MessageOptions\" is not defined.\nenvoy/admin/v3/certs.proto: Import \"google/protobuf/timestamp.proto\" was not found or had errors.\nenvoy/admin/v3/certs.proto: Import \"udpa/annotations/status.proto\" was not found or had errors.\nenvoy/admin/v3/certs.proto: Import \"udpa/annotations/versioning.proto\" was not found or had errors.\nenvoy/admin/v3/certs.proto:45:5: \"google.protobuf.Timestamp\" is not defined.\nenvoy/admin/v3/certs.proto:48:5: \"google.protobuf.Timestamp\" is not defined.\nenvoy/admin/v3/certs.proto:64:3: \"google.protobuf.Timestamp\" is not defined.\nenvoy/admin/v3/certs.proto:67:3: \"google.protobuf.Timestamp\" is not defined.\n" }
warning: build failed, waiting for other jobs to finish...

This is a good example of why https://github.com/kubernetes-sigs/gateway-api-inference-extension/pull/1313/files#r2316592252.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All required steps should be captured here instead of an external link

Note: Due to #1502, we plan on removing most 3rd party content from the docs site. We will instead refer to the implementations page. Conformant implementations will be required to maintain their own install guides.

Yeah, but that would potentially make this page even more unreadable. And even for GKE there is a link to external docs. Perhaps it is ok to proceed as is. Given that this is going to change anyway.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I followed the referenced readme but the ./run_kubvernor.sh step failed:

$ rustc --version
rustc 1.89.0 (29483883e 2025-08-04)
$ cargo version
cargo 1.89.0 (c24e10642 2025-06-23)
$ echo $CONTROL_PLANE_IP
172.18.0.2
$ ./run_kubvernor.sh
...
  --- stderr
  Error: Custom { kind: Other, error: "protoc failed: google/protobuf/timestamp.proto: File not found.\ngoogle/protobuf/descriptor.proto: File not found.\nudpa/annotations/status.proto: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\nudpa/annotations/status.proto:15:8: \"google.protobuf.FileOptions\" is not defined.\nudpa/annotations/versioning.proto: Import \"google/protobuf/descriptor.proto\" was not found or had errors.\nudpa/annotations/versioning.proto:13:8: \"google.protobuf.MessageOptions\" is not defined.\nenvoy/admin/v3/certs.proto: Import \"google/protobuf/timestamp.proto\" was not found or had errors.\nenvoy/admin/v3/certs.proto: Import \"udpa/annotations/status.proto\" was not found or had errors.\nenvoy/admin/v3/certs.proto: Import \"udpa/annotations/versioning.proto\" was not found or had errors.\nenvoy/admin/v3/certs.proto:45:5: \"google.protobuf.Timestamp\" is not defined.\nenvoy/admin/v3/certs.proto:48:5: \"google.protobuf.Timestamp\" is not defined.\nenvoy/admin/v3/certs.proto:64:3: \"google.protobuf.Timestamp\" is not defined.\nenvoy/admin/v3/certs.proto:67:3: \"google.protobuf.Timestamp\" is not defined.\n" }
warning: build failed, waiting for other jobs to finish...

This is a good example of why https://github.com/kubernetes-sigs/gateway-api-inference-extension/pull/1313/files#r2316592252.

Thanks, I have updated Kubvernor README.



3. Deploy the Gateway

```bash
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/kubvernor/gateway.yaml
```

Confirm that the Gateway was assigned an IP address and reports a `Programmed=True` status:
```bash
$ kubectl get gateway inference-gateway
NAME CLASS ADDRESS PROGRAMMED AGE
inference-gateway kubvernor-inference-gateway <MY_ADDRESS> True 22s
```

5. Deploy the HTTPRoute

```bash
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/gateway/kubvernor/httproute.yaml
```

6. Confirm that the HTTPRoute status conditions include `Accepted=True` and `ResolvedRefs=True`:

```bash
kubectl get httproute llm-route -o yaml
```



### Deploy the InferencePool and Endpoint Picker Extension

Expand Down Expand Up @@ -404,3 +440,6 @@ A cluster with:
```bash
kubectl delete ns kgateway-system
```
=== "Kubvernor"

No further clean up is needed.
7 changes: 7 additions & 0 deletions site-src/implementations/gateways.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,15 @@ This project has several implementations that are planned or in progress:
* [Google Kubernetes Engine][4]
* [Istio][5]
* [Kgateway][6]
* [Kubvernor Rust API Gateway][7]

[1]:#agentgateway
[2]:#alibaba-cloud-container-service-for-kubernetes
[3]:#envoy-ai-gateway
[4]:#google-kubernetes-engine
[5]:#istio
[6]:#kgateway
[7]:#kubvernor-rust-api-gateway

## Agentgateway

Expand Down Expand Up @@ -93,3 +95,8 @@ Issue](https://github.com/istio/istio/issues/55768).
gateway that can run [independently](https://gateway-api-inference-extension.sigs.k8s.io/guides/#__tabbed_3_3), as an [Istio waypoint](https://kgateway.dev/blog/extend-istio-ambient-kgateway-waypoint/),
or within your [llm-d infrastructure](https://github.com/llm-d-incubation/llm-d-infra) to improve accelerator (GPU)
utilization for AI inference workloads.

## Kubvernor Rust API Gateway
[Kubvernor Rust API Gateway][krg] is an open-source, highly experimental implementation of API controller in Rust programming language. Currently, Kubvernor supports Envoy Proxy. The project aims to be as generic as possible so Kubvernor can be used to manage/deploy different gateways (Envoy, Nginx, HAProxy, etc.). Kubvernor Rust API Gateway is conformant with Gateway-API-Inference-Extension v0.5.1.

[krg]:https://github.com/kubvernor/kubvernor