8000 Support starcoder family architectures (1B/3B/7B/13B) · Issue #3076 · ggerganov/llama.cpp · Issue #362 · irthomasthomas/undecidability · GitHub
[go: up one dir, main page]
Skip to content
/ undecidability Public
Support starcoder family architectures (1B/3B/7B/13B) · Issue #3076 · ggerganov/llama.cpp #362
New issue
New issue
Open
Open
Support starcoder family architectures (1B/3B/7B/13B) · Issue #3076 · ggerganov/llama.cpp#362
Labels
ModelsLLM and ML model repos and linksllmLarge Language Modelsllm-applicationsTopics related to practical applications of Large Language Models in various fieldsllm-evaluationEvaluating Large Language Models performance and behavior through human-written evaluation setsllm-inference-enginesSoftware to run inference on large language modelsllm-serving-optimisationsTips, tricks and tools to speedup inference of large language models
@irthomasthomas

Description

@irthomasthomas
irthomasthomas
opened on Jan 15, 2024

Previously, it wasn't recommended to incorporate non-llama architectures into llama.cpp. However, in light of the recent addition of the Falcon architecture (see Pull Request #2717), it might be worth reconsidering this stance.

One distinguishing feature of Starcoder is its ability to provide a complete series of models ranging from 1B to 13B. This capability can prove highly beneficial for speculative decoding and making coding models available for edge devices (e.g., M1/M2 Macs).

I can contribute the PR if it matches llama.cpp's roadmap.

Suggested labels

{ "key": "LLM-Applications", "value": "Practical applications of Large Language Models, such as edge device coding models and speculative decoding" } { "key": "Multimodal-LM", "value": "LLMs that combine modes such as text and image recognition" }

Metadata

Metadata

Assignees

No one assigned

    Labels

    ModelsLLM and ML model repos and linksllmLarge Language Modelsllm-applicationsTopics related to practical applications of Large Language Models in various fieldsllm-evaluationEvaluating Large Language Models performance and behavior through human-written evaluation setsllm-inference-enginesSoftware to run inference on large language modelsllm-serving-optimisationsTips, tricks and tools to speedup inference of large language models

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions
      0