Closed
Description
Short
In the Oobabooga installation, on Windows11, the package stops working when upgrading from 0.1.57 to 0.1.60. Reverting back to 0.1.57 fixes the problem. Complains about Shared library with base name 'llama' not found. There is no llama.dll anywhere in the folder, only llama.lib . I do not see any compilation errors.
Logs.
Installation log:
.\cmd_windows.bat
(F:\oobabooga_windows\installer_files\env) F:\oobabooga_windows>pip install llama-cpp-python --force-reinstall --upgrade --no-cache-dir
Collecting llama-cpp-python
Downloading llama-cpp-python-0.1.60.tar.gz (1.2 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.2/1.2 MB 516.0 kB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Installing backend dependencies ... done
Preparing metadata (pyproject.toml) ... done
Collecting typing-extensions>=4.5.0
Downloading typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Collecting numpy>=1.20.0
Downloading numpy-1.24.3-cp310-cp310-win_amd64.whl (14.8 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.8/14.8 MB 1.2 MB/s eta 0:00:00
Collecting diskcache>=5.6.1
Downloading diskcache-5.6.1-py3-none-any.whl (45 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 45.6/45.6 kB 2.2 MB/s eta 0:00:00
Building wheels for collected packages: llama-cpp-python
Building wheel for llama-cpp-python (pyproject.toml) ... done
Created wheel for llama-cpp-python: filename=llama_cpp_python-0.1.60-cp310-cp310-win_amd64.whl size=68815 sha256=3963c2f660e18df2a5e72cf2295cffd4d0c632accea654a06b31ee96cf6c5c52
Stored in directory: F:\oobabooga_windows\installer_files\pip-ephem-wheel-cache-1a8td5kq\wheels\eb\a7\7e\e2f6aaef90347cd520e86d37bf5c613d1b96eeab4512dc080a
Successfully built llama-cpp-python
Installing collected packages: typing-extensions, numpy, diskcache, llama-cpp-python
Attempting uninstall: typing-extensions
Found existing installation: typing_extensions 4.5.0
Uninstalling typing_extensions-4.5.0:
Successfully uninstalled typing_extensions-4.5.0
Attempting uninstall: numpy
Found existing installation: numpy 1.24.3
Uninstalling numpy-1.24.3:
Successfully uninstalled numpy-1.24.3
Attempting uninstall: diskcache
Found existing installation: diskcache 5.6.1
Uninstalling diskcache-5.6.1:
Successfully uninstalled diskcache-5.6.1
Attempting uninstall: llama-cpp-python
Found existing installation: llama-cpp-python 0.1.57
Uninstalling llama-cpp-python-0.1.57:
Successfully uninstalled llama-cpp-python-0.1.57
Successfully installed diskcache-5.6.1 llama-cpp-python-0.1.60 numpy-1.24.3 typing-extensions-4.6.3
Launch log
PS F:\oobabooga_windows> .\start_windows.bat
bin F:\oobabooga_windows\installer_files\env\lib\site-packages\bitsandbytes\libbitsandbytes_cuda117.dll
INFO:Loading settings from settings.yaml...
The following models are available:
1. facebook_opt-350m
2. gpt4-x-alpaca-native-13B-ggml-q5_1.bin
3. guanaco-13B.ggmlv3.q5_1.bin
4. Manticore-13B-Chat-Pyg-Guanaco-GGML-q4_0.bin
5. Manticore-13B.ggmlv3.q5_1.bin
6. Manticore-13B.ggmlv3.q5_K_M.bin
7. pygmalion-13b-ggml-q5_1.bin
8. Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_1.bin
9. WizardLM-13B-1.0.ggmlv3.q5_1.bin
10. WizardLM-Uncensored-SuperCOT-Storytelling.ggmlv3.q4_0.bin
Which one do you want to load? 1-10
3
INFO:Loading guanaco-13B.ggmlv3.q5_1.bin...
Traceback (most recent call last):
File "F:\oobabooga_windows\text-generation-webui\server.py", line 1079, in <module>
shared.model, shared.tokenizer = load_model(shared.model_name)
File "F:\oobabooga_windows\text-generation-webui\modules\models.py", line 94, in load_model
output = load_func(model_name)
File "F:\oobabooga_windows\text-generation-webui\modules\models.py", line 262, in llamacpp_loader
from modules.llamacpp_model import LlamaCppModel
File "F:\oobabooga_windows\text-generation-webui\modules\llamacpp_model.py", line 11, in <module>
from llama_cpp import Llama, LlamaCache
File "F:\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\__init__.py", line 1, in <module>
from .llama_cpp import *
File "F:\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\llama_cpp.py", line 77, in <module>
_lib = _load_shared_library(_lib_base_name)
File "F:\oobabooga_windows\installer_files\env\lib\site-packages\llama_cpp\llama_cpp.py", line 68, in _load_shared_library
raise FileNotFoundError(
FileNotFoundError: Shared library with base name 'llama' not found
Done!
Press any key to continue . . .
PS F:\oobabooga_windows>