I am testing out llamafile and have uploaded hermes here: https://huggingface.co/kevinbayes/Hermes-3-Llama-3.1-8B.llamafile

Goes without saying use cautiously.

@kevinbayes I downloaded your model but it throws errors:

gml_backend_cpu_buffer_type_alloc_buffer: failed to allocate buffer of size 17179869216
llama_kv_cache_init: failed to allocate buffer for kv cache
llama_new_context_with_model: llama_kv_cache_init() failed for self-attention cache
llama_init_from_gpt_params: error: failed to create context with model 'Hermes-3-Llama-3.1-8B.Q4_K_M.gguf'
main: error: unable to load model

I'm asking because I also have other 8B-Q4_K_M as llama file (and gguf) and they are loading correctly.

I consulted the error with Chatgpt and he told me about possible insufficient memory, but as I said I can load other models of the same size.

Could you please advice?

Thank you

@kevinbayes I have tested Hermes 3 on the website and i cannot seem to change it's censored alignment. I tried to make it swear but it won't do that. I. e. it's following someone else's alignment, not mine.