Closed
Description
Prerequisites
Please answer the following questions for yourself before submitting an issue.
- [X ] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
- [X ] I carefully followed the README.md.
- [X ] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
- [X ] I reviewed the Discussions, and have a new bug or useful enhancement to share.
Feature Description
Just add a CLI option in main.cpp in order to stop generation at newline token
Motivation
For some users like us, the goal is not to chat/have a discussion with a LLM, just to get a single line of output.
Possible Implementation
in main.cpp: eg:
if (params.stop_at_nl && output_tokens.size() > 0 && !embd.empty() && embd.back() == llama_token_nl(model)) {
break; // only generate 1 single output line
}