5 alternatives to llama.cpp

ggml-org/llama.cpp — LLM inference in C/C++ (133,192 stars, MIT, ai, ui, files).

These are the closest projects in the JustClaw-It index that we serve licence-cleared code from, ranked by relevance and by our code-quality score (1–10, the mean over each repo's graded components: completeness, code quality, deployability, reusability, strategic value). JustClaw-It also serves 1,798 licence-cleared components from ggml-org/llama.cpp itself.

The alternatives in detail

• nousresearch/nous-llama.cpp
  LLM inference in C/C++ - nous research
  score 4.9/10 · MIT permissive · 8 stars · 71 components · search them
• bentoml/bentoml
  The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
  score 7.7/10 · Apache-2.0 permissive · 8,917 stars · 230 components · search them
• bytedance/infinistore
  KV cache store for distributed LLM inference
  score 5.7/10 · Apache-2.0 permissive · 449 stars · 23 components · search them
• microsoft/llmlingua
  [EMNLP'23, ACL'24] To speed up LLMs' inference and enhance LLM's perceive of key information, compress the prompt and KV-Cache, which achieves up to 20x compression with minimal performance loss.
  score 4.9/10 · MIT permissive · 6,774 stars · 25 components · search them
• nousresearch/speculators
  A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
  score 5.0/10 · Apache-2.0 permissive · 3 stars · 77 components · search them

Licence data: JustClaw-It's per-repo licence policy. Is llama.cpp still maintained? Check it. Licence details: licence checker. Ranking data from 2026-11-05T20:19:43Z.