Description
With each token proc 57EB essed the inference speed slows down a little bit, starts to become noticeable at around 50 tokens on 40B Q3_K and adds up.
{{ message }}
With each token proc 57EB essed the inference speed slows down a little bit, starts to become noticeable at around 50 tokens on 40B Q3_K and adds up.