10000 gguf : make gguf pip-installable · CodeLinaro/llama.cpp@87e3733 · GitHub
[go: up one dir, main page]
Skip to content

Commit 87e3733

Browse files
monatismonatis
authored
gguf : make gguf pip-installable
* gitignore : add dist and rm pyproject.toml * gguf: prepare as Pip package * gguf: prepare as Pip package * gguf : fix line endings * requirements : add gguf * gguf : update readme with build notes * gguf : update readme with build notes * gguf : add notes for tests
1 parent b91ad7f commit 87e3733

File tree

8 files changed

+114
-1
lines changed

8 files changed

+114
-1
lines changed

‎.gitignore

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -60,6 +60,7 @@ compile_commands.json
6060
CMakeSettings.json
6161

6262
__pycache__
63+
dist
6364

6465
zig-out/
6566
zig-cache/
@@ -70,7 +71,6 @@ perf-*.txt
7071

7172
examples/jeopardy/results.txt
7273

73-
pyproject.toml
7474
poetry.lock
7575
poetry.toml
7676

‎gguf-py/LICENSE

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
MIT License
2+
3+
Copyright (c) 2023 Georgi Gerganov
4+
5+
Permission is hereby granted, free of charge, to any person obtaining a copy
6+
of this software and associated documentation files (the "Software"), to deal
7+
in the Software without restriction, including without limitation the rights
8+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9+
copies of the Software, and to permit persons to whom the Software is
10+
furnished to do so, subject to the following conditions:
11+
12+
The above copyright notice and this permission notice shall be included in all
13+
copies or substantial portions of the Software.
14+
15+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21+
SOFTWARE.

‎gguf-py/README.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
## gguf
2+
3+
This is a Python package for writing binary files in the [GGUF](https://github.com/ggerganov/ggml/pull/302)
4+
(GGML Universal File) format.
5+
6+
See [convert-llama-hf-to-gguf.py](https://github.com/ggerganov/llama.cpp/blob/master/convert-llama-hf-to-gguf.py)
7+
as an example for its usage.
8+
9+
## Installation
10+
```sh
11+
pip install gguf
12+
```
13+
14+
## Development
15+
Maintainers who participate in development of this package are advised to install it in editable mode:
16+
17+
```sh
18+
cd /path/to/llama.cpp/gguf-py
19+
20+
pip install --editable .
21+
```
22+
23+
**Note**: This may require to upgrade your Pip installation, with a message saying that editable installation currently requires `setup.py`.
24+
In this case, upgrade Pip to the latest:
25+
26+
```sh
27+
pip install --upgrade pip
28+
```
29+
30+
## Publishing
31+
To publish the package, you need to have `twine` and `build` installed:
32+
33+
```sh
34+
pip install build twine
35+
```
36+
37+
Then, folow these steps to release a new version:
38+
39+
1. Update the version in `pyproject.toml`.
40+
2. Build the package:
41+
42+
```sh
43+
python -m build
44+
```
45+
46+
3. Upload the generated distribution archives:
47+
48+
```sh
49+
python -m twine upload dist/*
50+
```
51+
52+
## TODO
53+
- [ ] Add tests
54+
- [ ] Include conversion scripts as command line entry points in this package.
55+
- Add CI workflow for releasing the package.

‎gguf-py/gguf/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
from .gguf import GGUFWriter

‎gguf.py renamed to ‎gguf-py/gguf/gguf.py

File renamed without changes.

‎gguf-py/pyproject.toml

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
[tool.poetry]
2+
name = "gguf"
3+
version = "0.2.0"
4+
description = "Write ML models in GGUF for GGML"
5+
authors = ["GGML <ggml@ggml.ai>"]
6+
packages = [
7+
{include = "gguf"},
8+
]
9+
readme = "README.md"
10+
homepage = "https://ggml.ai"
11+
repository = "https://github.com/ggerganov/llama.cpp"
12+
keywords = ["ggml", "gguf", "llama.cpp"]
13+
classifiers = [
14+
"Programming Language :: Python :: 3",
15+
"License :: OSI Approved :: MIT License",
16+
"Operating System :: OS Independent",
17+
]
18+
19+
[tool.poetry.dependencies]
20+
python = ">=3.8"
21+
numpy = ">=1.17"
22+
23+
[tool.poetry.dev-dependencies]
24+
pytest = "^5.2"
25+
26+
[build-system]
27+
requires = ["poetry-core>=1.0.0"]
28+
build-backend = "poetry.core.masonry.api"

‎gguf-py/tests/test_gguf.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
import gguf
2+
3+
# TODO: add tests
4+
5+
6+
def test_write_gguf():
7+
pass

‎requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,3 @@
11
numpy==1.24
22
sentencepiece==0.1.98
3+
gguf>=0.1.0

0 commit comments

Comments
 (0)
0