8000 New quantization method for Q4_K and Q5_K by Ttl · Pull Request #4739 · ggml-org/llama.cpp · GitHub
[go: up one dir, main page]
Skip to content

New quantization method for Q4_K and Q5_K #4739

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Ttl wants to merge 13 commits into from
Closed
8000

New quantization method for Q4_K and Q5_K #4739

Changes from 1 commit
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
433c4f5
Least squares quantization
Ttl Dec 26, 2023
0b6207e
Better k quantization
Ttl Dec 27, 2023
d1af0a3
Quantization loop
Ttl Dec 27, 2023
d6313d8
Refactor
Ttl Dec 27, 2023
f478136
Fix other than 4 quants
Ttl Dec 27, 2023
3fafeca
Weighted least squares
Ttl Dec 28, 2023
2a21bd8
Remove Q_K 0 type changes
Ttl Dec 28, 2023
8386034
Revert quants other than q4_k and q5_k
Ttl Dec 29, 2023
5a02328
No second least squares pass
Ttl Dec 30, 2023
4b06507
Cleanup
Ttl Jan 1, 2024
7e03872
Fix QKK_64
Ttl Jan 2, 2024
6683eb4
Fix buffer size
Ttl Jan 2, 2024
9c11f61
Simpler array sizes for msvc
Ttl Jan 2, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
Least squares quantization
  • Loading branch information
@Ttl
Ttl committed Dec 26, 2023
commit 433c4f505004afb0e60416f3c1514b2438f7c587
Loading
0