Colin Kealty retweeted
Join us in this livestream next Tuesday, learn how to shrink models to fit in your laptop and make them go brrr 💨
We will have @mervenoyann @maximelabonne @TadimetiAditya @bartowski1182 and @mgoin_ giving a primer and know-hows on quantization with llama.cpp and vLLM ⚡️
Introducing the Trinity Builders Program.
This is a community credit grant that gives developers, researchers, and open-source builders free inference access to Trinity models on our API.
Why we're doing it: Open weights only go so far if you don't have the compute to actually build with them. We want to lower that barrier so you can run experiments, iterate, and ship real systems.
Who this is for:
- Open-source projects
- Research initiatives
- Prototypes and production applications
- Developer tooling
Grants range from < 50M tokens to > 1B tokens, allocated based on your submitted project.
Submitting an application does not guarantee approval. We'll review requests on a best-effort basis, and allocation depends on compute capacity, token budgets, and other operational priorities.
We can't wait to see what the community builds with Trinity.
Learn more and apply here: arcee.ai/blog/introducing-th…
Today we're releasing Trinity-Large-Thinking.
Available now on the Arcee API, with open weights on Hugging Face under Apache 2.0.
We built it for developers and enterprises that want models they can inspect, post-train, host, distill, and own.
A banner weekend for Arcee as Trinity Large Preview officially entered the Top 5 overall models on @OpenRouter!
After surpassing 80B tokens on Sunday, we are already on pace to break that record again today. As our inference volume consistently tests the upper bound of our current limits, this growth signals a compounding market demand for our frontier-class open weights as we scale far beyond our initial 1T token milestone.
Stay tuned—more exciting news is on the horizon!
Colin Kealty retweeted
The usage from Trinity, especially while free on openrouter, has been incredible.
We’ve actually managed to handle 3x more throughput on our inference cluster than we estimated. But our TPS has tanked as usage just continues to grow.
We moved some training compute over to inference (not as simple as I thought it’d be) and we should be back to full speed.
Give us all you got.
Colin Kealty retweeted
DeepSeek-R1 sparked a revolution in RLVR, but moving to a stable Multi-GPU pipeline is an engineering minefield. 💣
I’m releasing "The Engineering Handbook for GRPO + LoRA with Verl", a guide to saturating A100s and building a industrial-grade RL loop.
Thanks to @maximelabonne for the feedbacks and insights! 🤝
Thread below. 🧵👇
Colin Kealty retweeted
Today, we are introducing Trinity, the start of an open-weight MoE family that businesses and developers can own.
Trinity-Mini (26B-A3B)
Trinity-Nano-Preview (6B-A1B)
Available Today on Huggingface.
Colin Kealty retweeted
🚀 LocalAI 3.8.0 is officially released!
We’ve just passed 39,000 stars 🌟 on GitHub. Thank you to the community!
This release is focused on removing any friction. We wanted to expose the full power of LocalAI multi-backend capabilities.
A deep dive into what’s new 🧵👇
Colin Kealty retweeted
🤗 @huggingface's most impactful contributors you should follow for the month of October is out! 🚀🚀 huggingface.co/spaces/TCTF/T…
🏛️ Top 10 Model likes: @TheBlokeAI, #mradermacher, @bartowski1182, #lllyasviel, #hexgrad, #nitrosocke, @kijaidesign, @haruu1367, @prompthero, #WarriorMama777
📊 Top 10 Dataset likes: @fkadev, @teknium, @maximelabonne, #gsdf, #Nerfgun3, @imhaotian, @OilandLuca, @logcong0120, @jon_durbin, #anon8231489123
🚀 Top 10 Space likes: @fffiloni, @multimodalart, @enzostvs, @flngr, @_akhaliq, @kingnish24, @xenovacom, @angrypenguinpng, @gokayfem, @yvrjsharma
🤗 Top 10 Profile followers: @TheBlokeAI, @lvminzhang, @mervenoyann, @bartowski1182, @_akhaliq, @ylecun, @fffiloni, @xenovacom, @maximelabonne, @Teknium
Congarts to all users 🔥🔥
Colin Kealty retweeted
Now in effect: Mergekit has been re-licensed under GNU LGPL v3, restoring clarity and flexibility for users and contributors. Read more about our decision in the blog.
arcee.ai/blog/mergekit-retur…
Colin Kealty retweeted
We’re going permissive: Apache 2.0 across the board. AFM-4.5B is now relicensed from Arcee to Apache 2.0; the agent variant will launch under Apache 2.0; and all upcoming releases ship with open weights. Three models are in training.
Colin Kealty retweeted
Meet Jan-v1-edge: an experimental 1.7B distilled model for Perplexity-style search.
Jan-v1-edge is our lightweight distillation experiment, derived from Jan v1. We're testing how well web search and reasoning can transfer into a smaller 1.7B parameter model that runs on edge devices.
Performance
- 83% SimpleQA accuracy, close to Jan-nano-128k while being lighter
- Outperforms Qwen3-1.7B Thinking on SimpleQA
To experiment with it, find the GGUF model on @huggingface, click Use this model and select Jan.
To enable search in Jan: go to Settings -> MCP Servers -> enable or add a search-related MCP (SearXNG, Serper, Exa, etc.).
- Jan-v1-edge: huggingface.co/janhq/Jan-v1-…
- Jan-v1-edge GGUF: huggingface.co/janhq/Jan-v1-…
Credit to the @Alibaba_Qwen team for Qwen3-1.7B Thinking and @ggerganov for llama.cpp.
Colin Kealty retweeted
As generative AI becomes increasingly central to business applications, the cost, complexity, and privacy concerns associated with language models are becoming significant.
At @arcee_ai, we’ve been asking a critical question: Can CPUs actually handle the demands of language models? The short answer: Yes—with the right optimizations.
In this piece by @AndrewWalko , @bartowski1182, and @julsimon , we explore hardware and software improvements that make CPU inference viable.
Using our latest foundation model, AFM-4.5B, we conduct detailed benchmarks on @intel Xeon, @awscloud Graviton, and @Qualcomm Z1E-80-100 CPUs to identify the most suitable configurations for enterprise AI use cases, whether in the cloud or at the edge.
Blog post: arcee.ai/blog/is-running-lan…
--
Follow @arcee_ai for the latest news on small language models.
Colin Kealty retweeted
this release is pure class. arcee using their data to do some short-term continued pretraining on GLM 32b.
long context support has gone from effectively 8k -> 32k, and all base model evaluations (including short context ones) have improved
Colin Kealty retweeted
Last week, we launched AFM-4.5B, our first foundation model.
In this post by @chargoddard , you will learn how we extended the context length of AFM-4.5B from 4k to 64k context through aggressive experimentation, model merging, distillation, and a concerning amount of soup.
Bon appétit 😋
Blog post: arcee.ai/blog/extending-afm-…
Colin Kealty retweeted
Today, we’re thrilled to unveil the @arcee_ai Foundation Models, a new family of GenAI models designed from the ground up for enterprise reality.
The first release—AFM-4.5B—is a 4.5-billion-parameter frontier model that delivers excellent accuracy, strict compliance, and very high cost efficiency.
In short: enterprise-grade intelligence that can run anywhere—on a smartphone, at the edge, or in the cloud.
For a quick taste, you can test AFM-4.5B in our playground and on Together.ai.
For a deeper dive into the model’s training pipeline and benchmarks, details are available in our technical blog post.
➡️ Arcee AI playground: afm.arcee.ai/
➡️ @togethercompute playground: api.together.ai/models/arcee…
➡️ Launch blog post: arcee.ai/blog/announcing-the…
➡️ Technical blog post: arcee.ai/blog/deep-dive-afm-…
PS: As we’ll now focus on our AFM foundation models, and because we love open-source, we're opening up access to our previously closed-source language models. Details in the tech blog post!
Colin Kealty retweeted
Vision support now available on llama.cpp server and Web UI!
More details in 🧵
Colin Kealty retweeted
idk when the official announcement is from our end but at this point i'm so grateful it's real idgaf @openrouter























