Skip to content

Automatically load available models for each provider. #85

@Art9681

Description

@Art9681

Currently, Manifold has hard coded models for the public providers, and the local provider models are provisioned manually by the user. The exception to this is Manifold will manage a local llama-server instance with a predefined GGUF model if the config is set to single node instance. We need to improve this process. The goals are:

  • Each public provider should automatically load the models available to the API key and list those in the relevant elements.
  • Implement a feature to manage local models. For now, llama.cpp and ideally mlx_lm.server. Manifold should enable a way to download models for the backends in the configured datapath, and list those models when the local provider is selected in the relevant elements.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions