8000 can't quant llama3 with expanded tokenizer · Issue #13628 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

can't quant llama3 with expanded tokenizer #13628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
SicariusSicariiStuff opened this issue May 19, 2025 · 0 comments
Open

can't quant llama3 with expanded tokenizer #13628

SicariusSicariiStuff opened this issue May 19, 2025 · 0 comments
Labels
bug-unconfirmed

Comments

@SicariusSicariiStuff
Copy link

Name and Version

latest llama.cpp won't quant llama3 with expanded bpe tokenizer (model works fine on fp16 and fp8 on aphrodite \ transformers \koboldcpp )

Operating systems

Linux

GGML backends

CUDA

Hardware

2xa6k

Models

llama 3.1

Problem description & steps to reproduce

Traceback (most recent call last):
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 5689, in
main()
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 5575, in main
model_instance.write()
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 441, in write
self.prepare_metadata(vocab_only=False)
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 434, in prepare_metadata
self.set_vocab()
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 1624, in set_vocab
self._set_vocab_gpt2()
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 746, in _set_vocab_gpt2
tokens, toktypes, tokpre = self.get_vocab_base()
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 527, in get_vocab_base
tokpre = self.get_vocab_base_pre(tokenizer)
File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 734, in get_vocab_base_pre
raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()")
NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre()

First Bad Commit

No response

Relevant log output

Traceback (most recent call last):
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 5689, in <module>
    main()
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 5575, in main
    model_instance.write()
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 441, in write
    self.prepare_metadata(vocab_only=False)
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 434, in prepare_metadata
    self.set_vocab()
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 1624, in set_vocab
    self._set_vocab_gpt2()
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 746, in _set_vocab_gpt2
    tokens, toktypes, tokpre = self.get_vocab_base()
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 527, in get_vocab_base
    tokpre = self.get_vocab_base_pre(tokenizer)
  File "/home/sicarius/llama.cpp/convert_hf_to_gguf.py", line 734, in get_vocab_base_pre
    raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()")
NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug-unconfirmed
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@SicariusSicariiStuff
0