RPC for qwen 72b? (unsupported quant) #10000
Unanswered
rubentorresbonet
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
No matter what quants I tried (Q4_0, Q4_K_M, ...) I always get the assert of unsupported quant from the RPC cpp file.
Do I need to select a very specific configuration or something for this to work?
Because the smaller model worked.
Thanks!
Beta Was this translation helpful? Give feedback.
All reactions