@ggml_org

AI inference at the edge

github.com/ggml-org
Joined April 2025
  • Tweets58
  • Following0
  • Followers4.2K
7Media
llama.cpp v0.7.0 github.com/ggml-org/llama.cp…

Release v0.7.0 · ggml-org/llama.cpp

Overview llama.cpp v0.7.0 adds the LiliCorr speculative decoding technique and MTP for GLM5-Next, a new GPU cache for MoE experts kept in host memory (--moe-cache-mib, up to ~2.2x decode speedup wh...

github.com
3
34
6
280
12,807
ggml v0.27.0 github.com/ggml-org/ggml/rel…

Release v0.27.0 · ggml-org/ggml

Overview ggml v0.27.0 extends the backend scheduler API with user callbacks for split weight copying and compute-buffer reallocation, and bumps the RPC protocol to v8 to support tensor-parallel RPC...

github.com
9
71
2,187
whisper.cpp v1.9.5 github.com/ggml-org/whisper.…

Release v1.9.5 · ggml-org/whisper.cpp

Overview Maintenance release. Updates to ggml v0.26.0. Nightly build: b5454 More info: dist : releases and versioning of ggml-org projects Changelog since v1.9.4 d1be6fd whisper : bump version to 1...

github.com
1
6
109
5,901
llama.cpp v0.6.0 github.com/ggml-org/llama.cp…

Release v0.6.0 · ggml-org/llama.cpp

Overview llama.cpp v0.6.0 introduces the new llama_batch_ext extended batch API (with llama_process) for mixed token/embedding inputs and MTP/deepstack state embeddings, adds support for the GLM-5....

github.com
6
38
11
342
81,997
ggml v0.26.0 github.com/ggml-org/ggml/rel…

Release v0.26.0 · ggml-org/ggml

Overview ggml v0.26.0 brings a new alloc_buffer_n/get_alloc_size_n buffer allocation API, sparse flash attention kernels on SYCL, Vulkan and Metal, and major lightning indexer improvements (halved ...

github.com
2
3
55
4,273
llama.cpp v0.5.0 github.com/ggml-org/llama.cp…

Release v0.5.0 · ggml-org/llama.cpp

Overview This release focuses on backend performance and correctness, broader model coverage, and more robust server/router operation. It adds HRM-Text (DFM Mimir 1B) support, MiMo-V2.6 and Hunyuan...

github.com
5
25
7
450
28,639
llama.cpp v0.4.1 github.com/ggml-org/llama.cp…

Release v0.4.1 · ggml-org/llama.cpp

Overview llama.cpp 0.4.1 adds Maple 20B-A1B, Tencent Hy 4, and Spark2.5 support, improves JSON schema handling, chat parsing, logging, and server child-process management, and updates ggml to v0.24...

github.com
10
29
3
210
17,870
ggml v0.24.0 github.com/ggml-org/ggml/rel…

Release v0.24.0 · ggml-org/ggml

Overview This release focuses on expanding backend coverage and robustness, with a new precision-control API, major Vulkan/SYCL/Hexagon/OpenCL work, and numerous correctness and performance fixes a...

github.com
1
24
1,735
llama.cpp v0.4.0 github.com/ggml-org/llama.cp…

Release v0.4.0 · ggml-org/llama.cpp

Overview llama.cpp 0.4.0 adds initial Qwen3.8-Flash-Next and Nemotron-3-Puzzle support, on-demand tensor reading, per-slot server context limits, video input options, and a ggml update to 0.23.0 wi...

github.com
4
60
7
483
31,734
ggml v0.23.0 github.com/ggml-org/ggml/rel…

Release v0.23.0 · ggml-org/ggml

Overview Version 0.23.0 adds new operators and backend scheduling APIs for sparse attention, asynchronous execution, and allocation-dependency tracking. It includes broad correctness and performanc...

github.com
1
40
3,025
llama.cpp v0.2.0 github.com/ggml-org/llama.cp…

Release v0.2.0 · ggml-org/llama.cpp

Overview This version marks the beginning of consistent semantic versioning for llama.cpp. The main goal is to make the job of downstream projects and users simpler when deciding which version of l...

github.com
8
41
4
507
35,077
whisper.cpp v1.8.5 github.com/ggml-org/whisper.…

Release v1.8.5 · ggml-org/whisper.cpp

Overview Maintenance release + performance improvements all around: https://github.com/ggml-org/whisper.cpp/blob/master/scripts/bench-all-gg.txt What's Changed ruby : fix dangling pointers, me...

github.com
1
1
37
1,680
whisper.cpp v1.8.4 github.com/ggml-org/whisper.…

Release v1.8.4 · ggml-org/whisper.cpp

Overview Maintenance release, latest ggml, some performance gains across the board. What's Changed Fix: UTF-8 character cut off to two "�" in segment wrapping (max_len) by @lordofriv...

github.com
1
26
2,363
ggml.ai joins Hugging Face github.com/ggml-org/llama.cp…
4
18
8
214
15,251
whisper.cpp v1.8.3 github.com/ggml-org/whisper.…

Release v1.8.3 · ggml-org/whisper.cpp

Overview Maintenance release, latest ggml, minor improvements in the tools/server/bindings. What's Changed server : set no_context == true by @ggerganov in #3482 WASM example broken for Hebrew...

github.com
3
1
1
43
9,311
github.com/ggml-org/llama.cp…

llama : add support for NVIDIA Nemotron 3 Nano by danbev · Pull Request #18058 · ggml-org/llama.cpp

This commit adds support for the NVIDIA Nemotron Nano 3 model, enabling the conversion and running of this model. Tech blog: https://developer.nvidia.com/blog/inside-nvidia-nemotron-3-techniques-t...

github.com
1
1
26
10,569
github.com/ggml-org/llama.cp…

llama: automatically set parameters not set by the user in such a way that maximizes GPU utilizat...

This PR adds automation for setting parameters in such a way that maximizes memory utilization when the full model cannot be fit. The short version is that the code first tries reducing the context...

github.com
1
2
20
2,454
github.com/ggml-org/llama.cp…

cli: new CLI experience by ngxson · Pull Request #17824 · ggml-org/llama.cpp

Ref: #17618 Fix: #11202 We are moving to a new CLI experience with the main code built on top of llama-server. This brings many additional features into llama-cli, making the experience feels mostl...

github.com
12
1,365
whisper.cpp v1.8.1 github.com/ggml-org/whisper.…

Release v1.8.1 · ggml-org/whisper.cpp

Overview Fix Vulkan builds Fix memory leaks when using VAD Support --carry-initial-prompt What's Changed bindings-java : disable flash attention by default by @danbev in #3445 ruby : Loose R...

github.com
7
75
4,002
github.com/ggml-org/llama.cp…

rpc : add support for multiple devices by rgerganov · Pull Request #16276 · ggml-org/llama.cpp

Allow rpc-server to expose multiple devices from a single endpoint. Change RPC protocol to include device identifier where needed. Add new API to get the device count from an RPC endpoint. closes: ...

github.com
8
1,166
Load more