Closed
Description
Hello,
I use this library to use llama-cpp in Python.
It's working very well but the generation is always the same (no seed in the script). I use this function
from llama_cpp import Llama
llama = Llama("./path/models/7B/ggml-model-7b.bin")
gen = llama.create_completion("Cite the best movie citation you know.",
max_tokens=50,
temperature=0.95,
top_p=0.95)I ran it several times and it's always the same text generated. Is there like HuggingFace a do_sample parameter to trigger ? I didn't find anything in the documentation or the issues of this github (but maybe I missed it)
Thank you, whoever you are who will help me :)