You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If you want to support cublas in the compilation, please make sure that you've installed it. If you are using Intel CPU, please check the highest AVX ([Advanced Vector Extensions](https://en.wikipedia.org/wiki/Advanced_Vector_Extensions)) level that is supported by your device.
30
30
31
-
As shown in [llama.cpp cmake file](https://github.com/ggerganov/llama.cpp/blob/master/CMakeLists.txt), there are many options that could be enabled or disabled when building the library. The following ones are commonly used when using it as a native library of LLamaSharp.
31
+
As shown in [llama.cpp cmake file](https://github.com/ggerganov/llama.cpp/blob/master/CMakeLists.txt), which also includes [llama.cpp/ggml cmake file](https://github.com/ggerganov/llama.cpp/blob/master/ggml/CMakeLists.txt), there are many options that could be enabled or disabled when building the library. The following ones are commonly used when using it as a native library of LLamaSharp.
32
32
33
33
```cpp
34
34
option(BUILD_SHARED_LIBS "build shared libraries") // Please always enable it
35
-
option(LLAMA_NATIVE "llama: enable -march=native flag") // Could be disabled
36
-
option(LLAMA_AVX "llama: enable AVX") // Enable it if the highest supported avx level is AVX
37
-
option(LLAMA_AVX2 "llama: enable AVX2") // Enable it if the highest supported avx level is AVX2
38
-
option(LLAMA_AVX512 "llama: enable AVX512") // Enable it if the highest supported avx level is AVX512
39
-
option(LLAMA_BLAS "llama: use BLAS") // Enable it if you want to use BLAS library to acclerate the computation on CPU
40
-
option(LLAMA_CUDA "llama: use CUDA") // Enable it if you have CUDA device
41
-
option(LLAMA_CLBLAST "llama: use CLBlast") // Enable it if you have a device with CLBLast or OpenCL support, for example, some AMD GPUs.
42
-
option(LLAMA_VULKAN "llama: use Vulkan") // Enable it if you have a device with Vulkan support
43
-
option(LLAMA_METAL "llama: use Metal") // Enable it if you are using a MAC with Metal device.
44
35
option(LLAMA_BUILD_TESTS "llama: build tests") // Please disable it.
45
36
option(LLAMA_BUILD_EXAMPLES "llama: build examples") // Please disable it.
46
37
option(LLAMA_BUILD_SERVER "llama: build server example")// Please disable it.
38
+
39
+
option(GGML_NATIVE "llama: enable -march=native flag") // Could be disabled
40
+
option(GGML_AVX "ggml: enable AVX") // Enable it if the highest supported avx level is AVX
41
+
option(GGML_AVX2 "ggml: enable AVX2") // Enable it if the highest supported avx level is AVX2
42
+
option(GGML_AVX512 "ggml: enable AVX512") // Enable it if the highest supported avx level is AVX512
43
+
option(GGML_CUDA "ggml: use CUDA") // Enable it if you have CUDA device
44
+
option(GGML_BLAS "ggml: use BLAS") // Enable it if you want to use BLAS library to acclerate the computation on CPU
45
+
option(GGML_VULKAN "ggml: use Vulkan") // Enable it if you have a device with Vulkan support
46
+
option(GGML_METAL "ggml: use Metal") // Enable it if you are using a MAC with Metal device.
47
47
```
48
48
49
-
Most importantly, `-DBUILD_SHARED_LIBS=ON` must be added to the cmake instruction and other options depends on you. For example, when building with cublas but without openblas, use the following instruction:
49
+
Most importantly, `-DBUILD_SHARED_LIBS=ON` must be added to the cmake instruction and other options depends on you. For example, when building with CUDA, use the following instruction:
50
50
51
51
```bash
52
52
mkdir build && cd build
53
-
cmake .. -DLLAMA_CUBLAS=ON -DBUILD_SHARED_LIBS=ON
53
+
cmake .. -DGGML_CUDA=ON -DBUILD_SHARED_LIBS=ON
54
54
cmake --build . --config Release
55
55
```
56
56
57
-
Now you could find the `llama.dll`, `libllama.so` or `llama.dylib` in your build directory (or `build/bin`).
57
+
Now you could find the `llama.dll`, `libllama.so` or `llama.dylib` in `build/src`.
58
58
59
59
To load the compiled native library, please add the following code to the very beginning of your code.
0 commit comments