abetlen llama-cpp-python · Discussions · GitHub

8000 abetlen llama-cpp-python · Discussions · GitHub

Sort by: Latest activity

Discussions

You must be logged in to vote

Hope to support dynamically loaded libraries ggml-base.so

yigekuyou started May 13, 2025 in Ideas

0
You must be logged in to vote

Running multiple tiny models in parallel on a single GPU

abarai-lanl asked May 11, 2025 in Q&A · Unanswered

0
You must be logged in to vote

How do I get <think> from e.g. Qwen 3 prompt with /think adds a paragraph with think.

LaKanDoR started May 2, 2025 in General

1
You must be logged in to vote

Is the bug template accurate when it says "there are no tagged versions as of now"?

nchammas asked Apr 10, 2025 in Q&A · Unanswered

0
You must be logged in to vote

Throughputs of Long Sequences #12608

simmonssong asked Mar 27, 2025 in Q&A · Unanswered

0
You must be logged in to vote

Delay between prompt submission and first token generation with longer prompts

jhthompson12 asked Dec 26, 2023 in Q&A · Unanswered

4
You must be logged in to vote

Super simple Python connectors for llama.cpp, including vision models (Gemma 3, Qwen2-VL)

fidecastro started Mar 19, 2025 in Show and tell

0
You must be logged in to vote

Use model for multiple user sessions

riebers-m started Sep 4, 2024 in General

9
You must be logged in to vote

Getting Embedding from CLIP model

lelefontaa asked Feb 18, 2025 in Q&A · Unanswered

0
You must be logged in to vote

Segmentation fault on tensor conversion

devashishraj asked Feb 15, 2025 in Q&A · Unanswered

0
You must be logged in to vote

Best way to apply chat templates locally

bwilkie asked Feb 10, 2025 in Q&A · Unanswered

0
You must be logged in to vote

THIS is how to Get CUDA Working with llama-cpp-python!

PasiKoodaa started Feb 9, 2025 in General · Closed

0
You must be logged in to vote

where is the UI file of the server locate in this python binding llama-cpp-python

heylobc started Feb 8, 2025 in General

0
You must be logged in to vote

OSError('exception: access violation reading 0x0000000000000000')

OreNot started Feb 5, 2025 in General

0
You must be logged in to vote

Api/generate endpoint

gl2007 asked Feb 5, 2025 in Q&A · Unanswered

0
You must be logged in to vote

How to make it work with CUDA-support?

itinance asked Aug 18, 2024 in Q&A · Unanswered

2
You must be logged in to vote

[GUIDE] How to ACTUALLY build with cuBLAS support on windows

tk-master started Nov 4, 2023 in General

14
You must be logged in to vote

Help with illegal hardware instruction

SpinNeutron asked Oct 21, 2023 in Q&A · Unanswered

3
You must be logged in to vote

Worse speed and GPU load than pure llama-cpp

Mushoz asked Nov 14, 2024 in Q&A · Answered

3
You must be logged in to vote

Numerous ggml_metal_init messages on init of class

astrowonk asked Dec 28, 2024 in Q&A · Unanswered

0
You must be logged in to vote

What need to be done to enable automatic function calling & "auto" tool choice?

samuelint asked Jul 23, 2024 in Q&A · Unanswered

6
You must be logged in to vote

Automating Pre-Building of Wheels for All Platforms (v0.3.4 and Beyond)

FarisZahrani started Dec 20, 2024 in Ideas · Closed

0
You must be logged in to vote

How to get the number of tokens of the prompt?

enzomich asked Dec 19, 2024 in Q&A · Unanswered

0
You must be logged in to vote

Generate Text with Embedding as Input

Sappique asked Dec 9, 2024 in Q&A · Unanswered

0
You must be logged in to vote

How to allow parallel inference requests

webhaypac asked Sep 3, 2024 in Q&A · Unanswered

4

0