8000 Segfault on load for embedding models lacking BOS/EOS token (such as `BAAI/bge-*`) · Issue #1463 · abetlen/llama-cpp-python · GitHub
[go: up one dir, main page]

Skip to content
Segfault on load for embedding models lacking BOS/EOS token (such as BAAI/bge-*) #1463
Closed
@iamlemec

Description

@iamlemec

For embedding models that lack BOS/EOS tokens (such as BAAI/bge-*), the BOS/EOS token ids default to -1, which causes a segfault on loading when calling token_get_text. I would recommend either short-circuiting these calls to the empty string in that case or skipping the chat template code entirely for embedding models.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      0