1
1
# 🦙 Python Bindings for ` llama.cpp `
2
2
3
+ [ ![ PyPI] ( https://img.shields.io/pypi/v/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
4
+ [ ![ PyPI - Python Version] ( https://img.shields.io/pypi/pyversions/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
5
+ [ ![ PyPI - License] ( https://img.shields.io/pypi/l/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
6
+ [ ![ PyPI - Downloads] ( https://img.shields.io/pypi/dm/llama-cpp-python )] ( https://pypi.org/project/llama-cpp-python/ )
7
+
3
8
Simple Python bindings for ** @ggerganov 's** [ ` llama.cpp ` ] ( https://github.com/ggerganov/llama.cpp ) library.
4
9
This package provides:
5
10
@@ -8,6 +13,42 @@ This package provides:
8
13
- OpenAI-like API
9
14
- LangChain compatibility
10
15
16
+ ## Installation
17
+
18
+ Install from PyPI:
19
+
20
+ ``` bash
21
+ pip install llama-cpp-python
22
+ ```
23
+
24
+ ## Usage
25
+
26
+ ``` python
27
+ >> > from llama_cpp import Llama
28
+ >> > llm = Llama(model_path = " models/7B/..." )
29
+ >> > output = llm(" Q: Name the planets in the solar system? A: " , max_tokens = 32 , stop = [" Q:" , " \n " ], echo = True )
30
+ >> > print (output)
31
+ {
32
+ " id" : " cmpl-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" ,
33
+ " object" : " text_completion" ,
34
+ " created" : 1679561337 ,
35
+ " model" : " models/7B/..." ,
36
+ " choices" : [
37
+ {
38
+ " text" : " Q: Name the planets in the solar system? A: Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune and Pluto." ,
39
+ " index" : 0 ,
40
+ " logprobs" : None ,
41
+ " finish_reason" : " stop"
42
+ }
43
+ ],
44
+ " usage" : {
45
+ " prompt_tokens" : 14 ,
46
+ " completion_tokens" : 28 ,
47
+ " total_tokens" : 42
48
+ }
49
+ }
50
+ ```
51
+
11
52
12
53
## API Reference
13
54
@@ -20,4 +61,8 @@ This package provides:
20
61
21
62
::: llama_cpp.llama_cpp
22
63
options:
23
- show_if_no_docstring: true
64
+ show_if_no_docstring: true
65
+
66
+ ## License
67
+
68
+ This project is licensed under the terms of the MIT license.
0 commit comments