8000 tts : fix n_ubatch + make WavTokenizer cache-less by ggerganov · Pull Request #13713 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

tts : fix n_ubatch + make WavTokenizer cache-less #13713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 22, 2025
Merged

tts : fix n_ubatch + make WavTokenizer cache-less #13713

merged 1 commit into from
May 22, 2025

Conversation

ggerganov
Copy link
Member

fix #13712

  • Set the WavTokenizer n_ubatch == n_batch since it is computing embeddings
  • Use llama_encode() instead of llama_decode() with the WavTokenizer context
  • The WavTokenizer does not need a KV cache, so we now no longer create it. This saves about ~800 MB of memory.

@ggerganov ggerganov mentioned this pull request May 22, 2025
Closed
@ggerganov ggerganov force-pushed the gg/tts-fix-ubatch branch from b814fb7 to 108d484 Compare May 22, 2025 18:58
@ggerganov ggerganov merged commit 8a1d206 into master May 22, 2025
51 of 53 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
examples
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Eval bug: llama-tts fails (abort) with longer lines
1 participant
@ggerganov
0