8000 Compile bug: There was a errror while compiling support for the backend Vulkan · Issue #12619 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

Compile bug: There was a errror while compiling support for the backend Vulkan #12619

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
mulinhu opened this issue Mar 28, 2025 · 10 comments
Open

Compile bug: There was a errror while compiling support for the backend Vulkan #12619

mulinhu opened this issue Mar 28, 2025 · 10 comments
Labels
bug-unconfirmed stale

Comments

@mulinhu
Copy link
mulinhu commented Mar 28, 2025

Git commit

git clone https://github.com/ggml-org/llama.cpp.git

Operating systems

Linux

GGML backends

Vulkan

Problem description & steps to reproduce

I'm trying to compile llama.cpp with vulkan support on a fresh install of orange pi 5 and get error.
Here are the exact commands that I used:

cmake -B build -DGGML_VULKAN=ON
cmake --build build

First Bad Commit

No response

Compile command

cmake -B build -DGGML_VULKAN=ON
cmake --build build

Relevant log output

(base) root@orangepiaipro:~/llama.cpp-master# lscpu
Architecture:          aarch64
  CPU op-mode(s):      64-bit
  Byte Order:          Little Endian
CPU(s):                4
  On-line CPU(s) list: 0-3
Vendor ID:             0x00
  Model:               2
  Thread(s) per core:  1
  Core(s) per cluster: 4
  Socket(s):           -
  Cluster(s):          1
  Stepping:            0x1
  BogoMIPS:            96.00
  Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics fphp asimdhp cpuid asimdrdm jscvt fcma dcpop sha3 asimddp sha512 sve asimdfhm ssbs sb
NUMA:                  
  NUMA node(s):        3
  NUMA node0 CPU(s):   0-3
  NUMA node1 CPU(s):   
  NUMA node2 CPU(s):   
Vulnerabilities:       
  Itlb multihit:       Not affected
  L1tf:                Not affected
  Mds:                 Not affected
  Meltdown:            Not affected
  Mmio stale data:     Not affected
  Retbleed:            Not affected
  Spec store bypass:   Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:          Mitigation; __user pointer sanitization
  Spectre v2:          Not affected
  Srbds:               Not affected
  Tsx async abort:     Not affected
(base) root@orangepiaipro:~/lyw/llama.cpp# cmake -B build -DGGML_VULKAN=ON
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.34.1") 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
-- ARM -mcpu not found, -mcpu=native will be used
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nodotprod
-- Performing Test GGML_MACHINE_SUPPORTS_nodotprod - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_noi8mm
-- Performing Test GGML_MACHINE_SUPPORTS_noi8mm - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_sve
-- Performing Test GGML_MACHINE_SUPPORTS_sve - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nosve
-- Performing Test GGML_MACHINE_SUPPORTS_nosve - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_sme
-- Performing Test GGML_MACHINE_SUPPORTS_sme - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nosme
-- Performing Test GGML_MACHINE_SUPPORTS_nosme - Failed
-- ARM feature FMA enabled
-- Adding CPU backend variant ggml-cpu: -mcpu=native 
-- Found Vulkan: /usr/lib/aarch64-linux-gnu/libvulkan.so  
-- 
8000
Vulkan found
-- GL_KHR_cooperative_matrix supported by glslc
-- GL_NV_cooperative_matrix2 supported by glslc
-- Including Vulkan backend
-- Configuring done
-- Generating done
-- Build files have been written to: /root/lyw/llama.cpp/build
(base) root@orangepiaipro:~/lyw/llama.cpp# cmake --build build
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-backend.cpp.o
[  2%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-opt.cpp.o
[  3%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/ggml-threading.cpp.o
[  3%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-quants.c.o
[  4%] Building CXX object ggml/src/CMakeFiles/ggml-base.dir/gguf.cpp.o
[  4%] Linking CXX shared library ../../bin/libggml-base.so
[  4%] Built target ggml-base
[  5%] Building CXX object ggml/src/ggml-vulkan/vulkan-shaders/CMakeFiles/vulkan-shaders-gen.dir/vulkan-shaders-gen.cpp.o
[  5%] Linking CXX executable ../../../../bin/vulkan-shaders-gen
[  5%] Built target vulkan-shaders-gen
[  5%] Generate vulkan shaders
ggml_vulkan: Generating and compiling shaders to SPIR-V
[  6%] Building CXX object ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘void ggml_vk_create_pipeline_func(vk_device&, vk_pipeline&, size_t, const void*, std::string, uint32_t, std::array<unsigned int, 3>, std::vector<unsigned int>, bool, bool, uint32_t)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:951:9: error: ‘PipelineRobustnessCreateInfoEXT’ is not a member of ‘vk’; did you mean ‘PipelineColorWriteCreateInfoEXT’?
  951 |     vk::PipelineRobustnessCreateInfoEXT rci;
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |         PipelineColorWriteCreateInfoEXT
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:954:9: error: ‘rci’ was not declared in this scope
  954 |         rci.storageBuffers = vk::PipelineRobustnessBufferBehaviorEXT::eDisabled;
      |         ^~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:954:34: error: ‘vk::PipelineRobustnessBufferBehaviorEXT’ has not been declared
  954 |         rci.storageBuffers = vk::PipelineRobustnessBufferBehaviorEXT::eDisabled;
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:955:34: error: ‘vk::PipelineRobustnessBufferBehaviorEXT’ has not been declared
  955 |         rci.uniformBuffers = vk::PipelineRobustnessBufferBehaviorEXT::eDisabled;
      |                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘vk_device ggml_vk_get_device(size_t)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2612:9: error: ‘VkPhysicalDevicePipelineRobustnessFeaturesEXT’ was not declared in this scope; did you mean ‘VkPhysicalDeviceImageRobustnessFeaturesEXT’?
 2612 |         VkPhysicalDevicePipelineRobustnessFeaturesEXT pl_robustness_features;
      |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |         VkPhysicalDeviceImageRobustnessFeaturesEXT
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2613:9: error: ‘pl_robustness_features’ was not declared in this scope
 2613 |         pl_robustness_features.pNext = nullptr;
      |         ^~~~~~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:2614:40: error: ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PIPELINE_ROBUSTNESS_FEATURES_EXT’ was not declared in this scope; did you mean ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_ROBUSTNESS_FEATURES_EXT’?
 2614 |         pl_robustness_features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_PIPELINE_ROBUSTNESS_FEATURES_EXT;
      |                                        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                                        VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_ROBUSTNESS_FEATURES_EXT
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘void ggml_vk_print_gpu_info(size_t)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:3008:5: error: ‘VkPhysicalDeviceCooperativeMatrixFeaturesKHR’ was not declared in this scope; did you mean ‘VkPhysicalDeviceCooperativeMatrixFeaturesNV’?
 3008 |     VkPhysicalDeviceCooperativeMatrixFeaturesKHR coopmat_features;
      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |     VkPhysicalDeviceCooperativeMatrixFeaturesNV
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:3009:5: error: ‘coopmat_features’ was not declared in this scope
 3009 |     coopmat_features.pNext = nullptr;
      |     ^~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:3010:30: error: ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_KHR’ was not declared in this scope; did you mean ‘VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_NV’?
 3010 |     coopmat_features.sType = VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_KHR;
      |                              ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      |                              VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_COOPERATIVE_MATRIX_FEATURES_NV
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘vk_pipeline ggml_vk_guess_matmul_pipeline(ggml_backend_vk_context*, vk_matmul_pipeline&, int, int, bool, ggml_type)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4025:54: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
 4025 |         if ((ctx->device->mul_mat_l[src0_type] && (n > crossover_large)) || (!ctx->device->mul_mat_m[src0_type] && !ctx->device->mul_mat_s[src0_type])) {
      |                                                    ~~^~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4030:54: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
 4030 |         if ((ctx->device->mul_mat_m[src0_type] && (n > crossover_medium)) || !ctx->device->mul_mat_s[src0_type]) {
      |                                                    ~~^~~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp: In function ‘vk_pipeline ggml_vk_guess_matmul_id_pipeline(ggml_backend_vk_context*, vk_matmul_pipeline&, int, int, bool, ggml_type)’:
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4081:57: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
 4081 |         if ((ctx->device->mul_mat_id_l[src0_type] && (n > crossover_large)) || (!ctx->device->mul_mat_id_m[src0_type] && !ctx->device->mul_mat_id_s[src0_type])) {
      |                                                       ~~^~~~~~~~~~~~~~~~~
/root/lyw/llama.cpp/ggml/src/ggml-vulkan/ggml-vulkan.cpp:4086:57: warning: comparison of integer expressions of different signedness: ‘int’ and ‘uint32_t’ {aka ‘unsigned int’} [-Wsign-compare]
 4086 |         if ((ctx->device->mul_mat_id_m[src0_type] && (n > crossover_medium)) || !ctx->device->mul_mat_id_s[src0_type]) {
      |                                                       ~~^~~~~~~~~~~~~~~~~~
gmake[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/build.make:184: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/ggml-vulkan.cpp.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:1808: ggml/src/ggml-vulkan/CMakeFiles/ggml-vulkan.dir/all] Error 2
@0cc4m
Copy link
Collaborator
0cc4m commented Mar 28, 2025

Your Vulkan headers are too old.

@mulinhu
Copy link
Author
mulinhu commented Mar 28, 2025

Your Vulkan headers are too old.

Do you know how to update the Vulkan headers?

@0cc4m
Copy link
Collaborator
0cc4m commented Mar 28, 2025

That depends on your system. Maybe it's possible to install a Vulkan SDK, or some PPA if it's Debian-based.

@mulinhu
Copy link
Author
mulinhu commented Mar 28, 2025

I don't know how to get the up-to-date Vulkan SDK version of linux with aarch64 architecture.

@mulinhu
Copy link
Author
mulinhu commented Mar 28, 2025

That depends on your system. Maybe it's possible to install a Vulkan SDK, or some PPA if it's Debian-based.

Do you know how to get the latest Vulkan SDK? My environment is ubuntu 22.04 and the cpu architecture is aarch64.

@ag2s20150909
Copy link
Contributor

https://vulkan.lunarg.com/sdk/home#linux

@0cc4m
Copy link
Collaborator
0cc4m commented Mar 28, 2025

https://vulkan.lunarg.com/sdk/home#linux

That's just for AMD64. But I don't know the best way for AARCH64, maybe the only option is building it yourself.

@ag2s20150909
Copy link
Contributor

https://github.com/RavEngine/vulkan-sdk-arm

@mulinhu
Copy link
Author
mulinhu commented Apr 22, 2025

Thank you very much!!!
I slove this problem using the vulkan-sdk-arm.

@jinwoo-enerzai
Copy link
jinwoo-enerzai commented Apr 22, 2025 •
edited
Loading

I just updated the github workflow so that it works on the current setup.
And you can download the artifact, feel free to use!

https://github.com/jinwoo-enerzai/vulkan-sdk-arm

@github-actions github-actions bot added the stale label May 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug-unconfirmed stale
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@0cc4m @ag2s20150909 @mulinhu @jinwoo-enerzai
0