-
Notifications
You must be signed in to change notification settings - Fork 26
Open
Labels
enhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed
Description
Currently, Manifold has hard coded models for the public providers, and the local provider models are provisioned manually by the user. The exception to this is Manifold will manage a local llama-server
instance with a predefined GGUF model if the config is set to single node instance. We need to improve this process. The goals are:
- Each public provider should automatically load the models available to the API key and list those in the relevant elements.
- Implement a feature to manage local models. For now,
llama.cpp
and ideallymlx_lm.server
. Manifold should enable a way to download models for the backends in the configured datapath, and list those models when the local provider is selected in the relevant elements.
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requesthelp wantedExtra attention is neededExtra attention is needed