Skip to content

Commit f0d4d17

Browse files
authored
Documentation: Update build.md's Vulkan section (#14736)
* Documentation: Rewrote and updated the "Without docker" portion of the Vulkan backend build documentation. * Documentation: Reorganize build.md's Vulkan section.
1 parent b172309 commit f0d4d17

File tree

1 file changed

+18
-22
lines changed

1 file changed

+18
-22
lines changed

docs/build.md

Lines changed: 18 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -305,9 +305,8 @@ On Linux it is possible to use unified memory architecture (UMA) to share main m
305305

306306
## Vulkan
307307

308-
**Windows**
309-
310-
### w64devkit
308+
### For Windows Users:
309+
**w64devkit**
311310

312311
Download and extract [`w64devkit`](https://github.com/skeeto/w64devkit/releases).
313312

@@ -334,7 +333,7 @@ cmake -B build -DGGML_VULKAN=ON
334333
cmake --build build --config Release
335334
```
336335

337-
### Git Bash MINGW64
336+
**Git Bash MINGW64**
338337

339338
Download and install [`Git-SCM`](https://git-scm.com/downloads/win) with the default settings
340339

@@ -357,7 +356,8 @@ Now you can load the model in conversation mode using `Vulkan`
357356
build/bin/Release/llama-cli -m "[PATH TO MODEL]" -ngl 100 -c 16384 -t 10 -n -2 -cnv
358357
```
359358

360-
### MSYS2
359+
**MSYS2**
360+
361361
Install [MSYS2](https://www.msys2.org/) and then run the following commands in a UCRT terminal to install dependencies.
362362
```sh
363363
pacman -S git \
@@ -373,9 +373,9 @@ cmake -B build -DGGML_VULKAN=ON
373373
cmake --build build --config Release
374374
```
375375

376-
**With docker**:
376+
### For Docker users:
377377

378-
You don't need to install Vulkan SDK. It will be installed inside the container.
378+
You don't need to install the Vulkan SDK. It will be installed inside the container.
379379

380380
```sh
381381
# Build the image
@@ -385,32 +385,28 @@ docker build -t llama-cpp-vulkan --target light -f .devops/vulkan.Dockerfile .
385385
docker run -it --rm -v "$(pwd):/app:Z" --device /dev/dri/renderD128:/dev/dri/renderD128 --device /dev/dri/card1:/dev/dri/card1 llama-cpp-vulkan -m "/app/models/YOUR_MODEL_FILE" -p "Building a website can be done in 10 simple steps:" -n 400 -e -ngl 33
386386
```
387387

388-
**Without docker**:
388+
### For Linux users:
389389

390-
Firstly, you need to make sure you have installed [Vulkan SDK](https://vulkan.lunarg.com/doc/view/latest/linux/getting_started_ubuntu.html)
390+
First, follow the the official [Getting Started with the Linux Tarball Vulkan SDK](https://vulkan.lunarg.com/doc/sdk/latest/linux/getting_started.html) guide.
391391

392-
For example, on Ubuntu 22.04 (jammy), use the command below:
392+
> [!IMPORTANT]
393+
> After completing the first step, ensure that you have used the `source` command on the `setup_env.sh` file inside of the Vulkan SDK in your current terminal session. Otherwise, the build won't work. Additionally, if you close out of your terminal, you must perform this step again if you intend to perform a build. However, there are ways to make this persistent. Refer to the Vulkan SDK guide linked in the first step for more information about any of this.
393394
395+
Second, after verifying that you have done everything in the Vulkan SDK guide provided in the first step, run the following command to verify that everything is set up correctly:
394396
```bash
395-
wget -qO - https://packages.lunarg.com/lunarg-signing-key-pub.asc | apt-key add -
396-
wget -qO /etc/apt/sources.list.d/lunarg-vulkan-jammy.list https://packages.lunarg.com/vulkan/lunarg-vulkan-jammy.list
397-
apt update -y
398-
apt-get install -y vulkan-sdk
399-
# To verify the installation, use the command below:
400397
vulkaninfo
401398
```
402399

403-
Alternatively your package manager might be able to provide the appropriate libraries.
404-
For example for Ubuntu 22.04 you can install `libvulkan-dev` instead.
405-
For Fedora 40, you can install `vulkan-devel`, `glslc` and `glslang` packages.
406-
407-
Then, build llama.cpp using the cmake command below:
408-
400+
Then, assuming you have `cd` into your llama.cpp folder and there are no errors with running `vulkaninfo`, you can proceed to build llama.cpp using the CMake commands below:
409401
```bash
410402
cmake -B build -DGGML_VULKAN=1
411403
cmake --build build --config Release
404+
```
405+
406+
Finally, after finishing your build, you should be able to do this:
407+
```bash
412408
# Test the output binary (with "-ngl 33" to offload all layers to GPU)
413-
./bin/llama-cli -m "PATH_TO_MODEL" -p "Hi you how are you" -n 50 -e -ngl 33 -t 4
409+
./build/bin/llama-cli -m "PATH_TO_MODEL" -p "Hi you how are you" -n 50 -e -ngl 33 -t 4
414410

415411
# You should see in the output, ggml_vulkan detected your GPU. For example:
416412
# ggml_vulkan: Using Intel(R) Graphics (ADL GT2) | uma: 1 | fp16: 1 | warp size: 32

0 commit comments

Comments
 (0)