You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
### Deploy the InferencePool and Endpoint Picker Extension
87
+
88
+
Install an InferencePool named `vllm-llama3-8b-instruct` that selects from endpoints with label `app: vllm-llama3-8b-instruct` and listening on port 8000. The Helm install command automatically installs the endpoint-picker, inferencepool along with provider specific resources.
Choose one of the following options to deploy an Inference Gateway.
@@ -268,22 +319,6 @@ A cluster with:
268
319
kubectl get httproute llm-route -o yaml
269
320
```
270
321
271
-
272
-
### Deploy the InferencePool and Endpoint Picker Extension
273
-
274
-
Install an InferencePool named `vllm-llama3-8b-instruct` that selects from endpoints with label app: vllm-llama3-8b-instruct and listening on port 8000, you can run the following command:
275
-
276
-
```bash
277
-
export GATEWAY_PROVIDER=none # See [README](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/config/charts/inferencepool/README.md#configuration) for valid configurations
0 commit comments