Closed
Description
Git commit
git clone https://github.com/ggml-org/llama.cpp.git
Operating systems
Linux
GGML backends
Vulkan
Problem description & steps to reproduce
I'm trying to compile llama.cpp with vulkan support on a fresh install of orange pi 5 and get error.
Here are the exact commands that I used:
cmake -B build -DGGML_VULKAN=ON
cmake --build build
First Bad Commit
No response
Compile command
cmake -B build -DGGML_VULKAN=ON
cmake --build buildRelevant log output
(base) root@orangepiaipro:~/llama.cpp-master# lscpu
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Vendor ID: 0x00
Model: 2
Thread(s) per core: 1
Core(s) per cluster: 4
Socket(s): -
Cluster(s): 1
Stepping: 0x1
BogoMIPS: 96.00
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop sha3 asimddp sha512 sve asimdfhm ssbs sb
NUMA:
NUMA node(s): 3
NUMA node0 CPU(s): 0-3
NUMA node1 CPU(s):
NUMA node2 CPU(s):
Vulnerabilities:
Itlb multihit: Not affected
L1tf: Not affected
Mds: Not affected
Meltdown: Not affected
Mmio stale data: Not affected
Retbleed: Not affected
Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl
Spectre v1: Mitigation; __user pointer sanitization
Spectre v2: Not affected
Srbds: Not affected
Tsx async abort: Not affected
(base) root@orangepiaipro:~/lyw/llama.cpp# cmake -B build -DGGML_VULKAN=ON
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1")
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- ARM -mcpu not found, -mcpu=native will be used
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nodotprod
-- Performing Test GGML_MACHINE_SUPPORTS_nodotprod - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_noi8mm
-- Performing Test GGML_MACHINE_SUPPORTS_noi8mm - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_sve
-- Performing Test GGML_MACHINE_SUPPORTS_sve - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nosve
-- Performing Test GGML_MACHINE_SUPPORTS_nosve - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_sme
-- Performing Test GGML_MACHINE_SUPPORTS_sme - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nosme
-- Performing Test GGML_MACHINE_SUPPORTS_nosme - Failed
-- ARM feature FMA enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=native
-- Found Vulkan: /usr/lib/aarch64-linux-gnu/libvulkan.so
-- Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Including Vulkan backend
-- Configuring done
-- Generating done
-- Build files have been written to: /root/lyw/llama.cpp/build
(base) root@orangepiaipro:~/lyw/llama.cpp# cmake --build build
[ 1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[ 1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[ 2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[ 2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[ 3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[ 3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[ 4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[ 4%] Linking CXX shared library ../../bin/libggml-base.so
[ 4%] Built target ggml-base
[ 5%] Building CXX object ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[ 5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[ 5%] Built target vulkan-shaders-gen
[ 5%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[ 6%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘void ggml_vk_create_pipeline_func(vk_device&, vk_pipeline&, size_t, const void*, std::string, uint32_t, std::array<unsigned int, 3>, std::vector<unsigned int>, bool, bool, uint32_t)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:951:9: error: ‘PipelineRobustnessCreateInfoEXT’ is not a member of ‘vk’; did you mean ‘PipelineColorWriteCreateInfoEXT’?
951 | vk::PipelineRobustnessCreateInfoEXT rci;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| PipelineColorWriteCreateInfoEXT
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:954:9: error: ‘rci’ was not declared in this scope
954 | rci.storageBuffers = vk::PipelineRobustnessBufferBehaviorEXT::eDisabled;
| ^~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:954:34: error: ‘vk::PipelineRobustnessBufferBehaviorEXT’ has not been declared
954 | rci.storageBuffers = vk::PipelineRobustnessBufferBehaviorEXT::eDisabled;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:955:34: error: ‘vk::PipelineRobustnessBufferBehaviorEXT’ has not been declared
955 | rci.uniformBuffers = vk::PipelineRobustnessBufferBehaviorEXT::eDisabled;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘vk_device ggml_vk_get_device(size_t)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2612:9: error: ‘VkPhysicalDevicePipelineRobustnessFeaturesEXT’ was not declared in this scope; did you mean ‘VkPhysicalDeviceImageRobustnessFeaturesEXT’?
2612 | VkPhysicalDevicePipelineRobustnessFeaturesEXT pl_robustness_features;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| VkPhysicalDeviceImageRobustnessFeaturesEXT
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2613:9: error: ‘pl_robustness_features’ was not declared in this scope
2613 | pl_robustness_features.pNext = nullptr;
| ^~~~~~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2614:40: error: ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PIPELINE_ROBUSTNESS_FEATURES_EXT’ was not declared in this scope; did you mean ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_ROBUSTNESS_FEATURES_EXT’?
2614 | pl_robustness_features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PIPELINE_ROBUSTNESS_FEATURES_EXT;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_ROBUSTNESS_FEATURES_EXT
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘void ggml_vk_print_gpu_info(size_t)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:3008:5: error: ‘VkPhysicalDeviceCooperativeMatrixFeaturesKHR’ was not declared in this scope; did you mean ‘VkPhysicalDeviceCooperativeMatrixFeaturesNV’?
3008 | VkPhysicalDeviceCooperativeMatrixFeaturesKHR coopmat_features;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| VkPhysicalDeviceCooperativeMatrixFeaturesNV
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:3009:5: error: ‘coopmat_features’ was not declared in this scope
3009 | coopmat_features.pNext = nullptr;
| ^~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:3010:30: error: ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_KHR’ was not declared in this scope; did you mean ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_NV’?
3010 | coopmat_features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_KHR;
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_NV
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘vk_pipeline ggml_vk_guess_matmul_pipeline(ggml_backend_vk_context*, vk_matmul_pipeline&, int, int, bool, ggml_type)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4025:54: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
4025 | if ((ctx->device->mul_mat_l[src0_type] && (n > crossover_large)) || (!ctx->device->mul_mat_m[src0_type] && !ctx->device->mul_mat_s[src0_type])) {
| ~~^~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4030:54: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
4030 | if ((ctx->device->mul_mat_m[src0_type] && (n > crossover_medium)) || !ctx->device->mul_mat_s[src0_type]) {
| ~~^~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘vk_pipeline ggml_vk_guess_matmul_id_pipeline(ggml_backend_vk_context*, vk_matmul_pipeline&, int, int, bool, ggml_type)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4081:57: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
4081 | if ((ctx->device->mul_mat_id_l[src0_type] && (n > crossover_large)) || (!ctx->device->mul_mat_id_m[src0_type] && !ctx->device->mul_mat_id_s[src0_type])) {
| ~~^~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4086:57: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
4086 | if ((ctx->device->mul_mat_id_m[src0_type] && (n > crossover_medium)) || !ctx->device->mul_mat_id_s[src0_type]) {
| ~~^~~~~~~~~~~~~~~~~~
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:184: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1808: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2