8000 server: do not remove whitespace at the start of a completion chunk (reapply to new UI) by mgroeber9110 · Pull Request #7830 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

server: do not remove whitespace at the start of a completion chunk (reapply to new UI) #7830

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Jun 9, 2024
Merged

server: do not remove whitespace at the start of a completion chunk (reapply to new UI) #7830

merged 1 commit into from
Jun 9, 2024

Conversation

mgroeber9110
Copy link
Contributor

This merges the change in #7524 to the new UI to keep the two implementations in sync.

Original description:

When using Completion mode of the server in recent builds, the first token of the predicted text is typically appended to the prompt without a space character. This can lead to problems when manually editing the text and then predicting again, because the missing space will appear in the new prompt.

When looking at this change again, I noticed that the code is also used in Chat mode, but in this case HTML rendering removes the whitespace at the start of a conversation turn (after "assistant"), and the conversation transcript looks to be okay afterwards.

@mofosyne
Copy link
Collaborator
mofosyne commented Jun 9, 2024

@mgroeber9110 is the CI issue related to your code or master branch?

@mgroeber9110
Copy link
Contributor Author

Looking at the error log, this appears to be related to the test environment for Windows:

      Traceback (most recent call last):
        File "C:\hostedtoolcache\windows\Python\3.11.9\x64\Lib\asyncio\windows_events.py", line 494, in finish_recv
          return ov.getresult()
                 ^^^^^^^^^^^^^^
      OSError: [WinError 64] The specified network name is no longer available

This looks unrelated to my change.

@mofosyne
Copy link
Collaborator
mofosyne commented Jun 9, 2024

Okay, then it's most likely due the issue a few weeks ago. Going to merge in directly anyway
892C

@mofosyne mofosyne merged commit 3e2ee44 into ggml-org:master Jun 9, 2024
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mofosyne mofosyne mofosyne approved these changes

Assignees
No one assigned
Labels
examples server
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@mgroeber9110 @mofosyne
0