Skip to content

feat: do not bundle llama-cpp anymore #5790

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 38 commits into from
Jul 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
630edfe
Build llama.cpp separately
mudler Jul 3, 2025
d0180e2
WIP
mudler Jul 4, 2025
41b2ddf
WIP
mudler Jul 7, 2025
cfe5719
WIP
mudler Jul 8, 2025
146a883
Start to try to attach some tests
mudler Jul 9, 2025
2e95742
Add git and small fixups
mudler Jul 9, 2025
e1783cd
fix: correctly autoload external backends
mudler Jul 11, 2025
8e81860
Try to run AIO tests
mudler Jul 11, 2025
10a03f1
Slightly update the Makefile helps
mudler Jul 11, 2025
9cfd148
Adapt auto-bumper
mudler Jul 11, 2025
58bba27
Try to run linux test
mudler Jul 11, 2025
90307ce
Add llama-cpp into build pipelines
mudler Jul 11, 2025
3548163
Add default capability (for cpu)
mudler Jul 11, 2025
972eaa5
Drop llama-cpp specific logic from the backend loader
mudler Jul 11, 2025
7a7051c
drop grpc install in ci for tests
mudler Jul 11, 2025
2cbbda7
fixups
mudler Jul 13, 2025
b70d7de
Pass by backends path for tests
mudler Jul 13, 2025
53c4b64
Build protogen at start
mudler Jul 13, 2025
362a603
fix(tests): set backends path consistently
mudler Jul 13, 2025
c78258b
Correctly configure the backends path
mudler Jul 14, 2025
a72b7f7
Try to build for darwin
mudler Jul 14, 2025
8f4e5b2
WIP
mudler Jul 15, 2025
7b319db
Compile for metal on arm64/darwin
mudler Jul 15, 2025
3e68ba5
Try to run build off from cross-arch
mudler Jul 15, 2025
06ca2d8
Add to the backend index nvidia-l4t and cpu's llama-cpp backends
mudler Jul 16, 2025
d8ebdee
Build also darwin-x86 for llama-cpp
mudler Jul 16, 2025
bfe4cd9
Disable arm64 builds temporary
mudler Jul 16, 2025
fecac3f
Test backend build on PR
mudler Jul 16, 2025
fe37822
Fixup build backend reusable workflow
mudler Jul 16, 2025
60e0dbe
pass by skip drivers
mudler Jul 16, 2025
63b860e
Use crane
mudler Jul 16, 2025
58426a0
Skip drivers
mudler Jul 16, 2025
5c1802a
Fixups
mudler Jul 16, 2025
b97c067
x86 darwin
mudler Jul 17, 2025
3891f31
Add packaging step for llama.cpp
mudler Jul 17, 2025
88fb292
fixups
mudler Jul 17, 2025
535689a
Fix leftover from bark-cpp extraction
mudler Jul 17, 2025
feecf58
Try to fix hipblas build
mudler Jul 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,9 @@
.vscode
.devcontainer
models
backends
examples/chatbot-ui/models
backend/go/image/stablediffusion-ggml/build/
examples/rwkv/models
examples/**/models
Dockerfile*
Expand All @@ -14,4 +16,4 @@ __pycache__

# backend virtual environments
**/venv
backend/python/**/source
backend/python/**/source
9 changes: 7 additions & 2 deletions .github/bump_deps.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,20 @@ set -xe
REPO=$1
BRANCH=$2
VAR=$3
FILE=$4

if [ -z "$FILE" ]; then
FILE="Makefile"
fi

LAST_COMMIT=$(curl -s -H "Accept: application/vnd.github.VERSION.sha" "https://api.github.com/repos/$REPO/commits/$BRANCH")

# Read $VAR from Makefile (only first match)
set +e
CURRENT_COMMIT="$(grep -m1 "^$VAR?=" Makefile | cut -d'=' -f2)"
CURRENT_COMMIT="$(grep -m1 "^$VAR?=" $FILE | cut -d'=' -f2)"
set -e

sed -i Makefile -e "s/$VAR?=.*/$VAR?=$LAST_COMMIT/"
sed -i $FILE -e "s/$VAR?=.*/$VAR?=$LAST_COMMIT/"

if [ -z "$CURRENT_COMMIT" ]; then
echo "Could not find $VAR in Makefile."
Expand Down
Loading
Loading