Can not using the ollama in docker container. ERROR: [LLM] http error #99
Description
It is a great plugin and I love it. But I found an error here.
[LLM] http error: error sending request for url (http://localhost:11434/api/generate): connection closed before message completed
Following the config in the readme.
{
"huggingface/llm.nvim",
opts = {
-- cf Setup
},
config = function()
local llm = require("llm")
llm.setup({
api_token = nil, -- cf Install paragraph
-- for ollama backend
backend = "ollama", -- backend ID, "huggingface" | "" | "openai" | "tgi"
model = "starcoder2:7b",
url = "http://localhost:11434/api/generate",
tokens_to_clear = { "<|endoftext|>" }, -- tokens to remove from the model's output
-- parameters that are added to the request body, values are arbitrary, you can set any field:value pair here it will be passed as is to the backend
request_body = {
parameters = {
max_new_tokens = 60,
temperature = 0.2,
top_p = 0.95,
},
},
-- set this if the model supports fill in the middle
fim = {
enabled = true,
prefix = "<fim_prefix>",
middle = "<fim_middle>",
suffix = "<fim_suffix>",
},
debounce_ms = 150,
accept_keymap = "<C-y>",
dismiss_keymap = "<C-n>",
tls_skip_verify_insecure = false,
-- llm-ls configuration, cf llm-ls section
lsp = {
bin_path = nil,
host = nil,
port = nil,
version = "0.5.2",
},
tokenizer = {
repository = "bigcode/starcoder2-7b",
}, -- cf Tokenizer paragraph
-- tokenizer = nil, -- cf Tokenizer paragraph
context_window = 4096, -- max number of tokens for the context window
enable_suggestions_on_startup = true,
enable_suggestions_on_files = "*", -- pattern matching syntax to enable suggestions on specific files, either a string or a list of strings
})
end,
}
The MOST wirred thing is that I can curl the answer to the same model & api url, and my vscode continue plugin can communicate this is ollama which is running on a docker container but this plugin cannot!
Thank you for your time and reply!