Skip to content

Cleanup uses of OpenAIMixin, simplify inference adapters #3517

@mattf

Description

@mattf

🤔 What is the technical debt you think should be addressed?

Provider chat completions completions embedding models: register models: listing models: availability models: refresh models: allowed provider-data: key config: key config: url health
anthropic mixin mixin mixin mixin mixin mixin mixin mixin mixin ANTHROPIC_API_KEY n/a no
azure openai mixin mixin mixin mixin mixin mixin mixin mixin mixin AZURE_API_KEY AZURE_API_BASE, /openai/v1 added in provider no
bedrock - #3748 custom (for telemetry and retry) mixin mixin custom (accept all) custom (not available) custom (accept all) mixin mixin mixin AWS_BEDROCK_API_KEY https://bedrock-runtime.{AWS_DEFAULT_REGION}.amazonaws.com/openai/v1 no
cerebras mixin mixin disabled mixin mixin mixin mixin mixin #3734 CEREBRAS_API_KEY coded in config, /v1 added in provider no
databricks mixin disabled mixin mixin mixin (not async) mixin mixin mixin #3734 DATABRICKS_TOKEN DATABRICKS_HOST, extended as needed in provider no
fireworks mixin mixin mixin mixin mixin mixin mixin mixin mixin FIREWORKS_API_KEY coded in config and provider no
gemini mixin mixin mixin mixin mixin mixin mixin mixin mixin GEMINI_API_KEY coded in provider no
groq mixin mixin mixin mixin mixin mixin mixin mixin mixin GROQ_API_KEY coded in config, /openai/v1 added in provider no
llama api mixin disabled disabled mixin mixin mixin mixin mixin mixin LLAMA_API_KEY coded in config no
nvidia mixin mixin custom (for input_type) mixin mixin mixin mixin mixin #3734 NVIDIA_API_KEY NVIDIA_BASE_URL, /v1 added in provider no
ollama mixin mixin mixin custom (for :latest) mixin mixin mixin mixin n/a n/a OLLAMA_URL w/ default, /v1 added yes
openai mixin mixin mixin mixin mixin mixin mixin mixin mixin OPENAI_API_KEY OPENAI_BASE_URL no
runpod custom (for stream usage) mixin mixin mixin mixin mixin mixin mixin #3734 RUNPOD_API_TOKEN RUNPOD_URL (no default, no mods) no
sambanova mixin mixin mixin mixin mixin mixin mixin mixin mixin SAMBANOVA_API_KEY coded in config no
tgi mixin mixin disabled mixin mixin mixin mixin mixin n/a n/a TGI_URL no
together mixin mixin custom (for api incompatibility) mixin mixin mixin mixin mixin mixin TOGETHER_API_KEY coded in config no
vertexai mixin mixin mixin mixin mixin mixin mixin mixin mixin GOOGLE_APPLICATION_CREDENTIALS provider constructed from VERTEX_AI_PROJECT & VERTEX_AI_LOCATION no
vllm custom (for tool_choice) mixin mixin mixin mixin mixin mixin mixin mixin VLLM_API_TOKEN w/ fake default VLLM_URL yes
watsonx - #3674 litellm litellm litellm litellm custom custom (always True) default no litellm WATSONX_API_KEY WATSONX_BASE_URL w/ default, also WATSONX_PROJECT_ID no

TODO: hf::serverless, hf::endpoints

💡 What is the benefit of addressing this technical debt?

maintainability.

Other thoughts

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions