AI, Local LLMs & Tech | Running AI models and agents at home | Problem solver. Don’t just doomscroll, build something! DMs open https://teapawt.net/t.co/diqJl9RaKj
Joined May 2016
- Tweets2.5K
- Following411
- Followers201
- Likes2.3K
Pinned Tweet
Got into running AI models on my own hardware, testing, building agents, and sharing what actually works. Follow for daily insights, hardware tips, and no-BS tech talk (+ an additional rant from time to time).
OpenAI's agents found 948 Linux priv-esc bugs in about 30 hours, cost ~$1.2 mil.
Great for security.
Just remember the same class of tools can be pointed at your stack.
Patch fast and keep sensitive boxes on LAN as much as possible, not like those guys with the hacked freezers.
RicursiveAI just released Ripple-65B, a dense 65B open-weight model claiming 80.5 on TerminalBench.
Not another trillion-param MoE. Something you can actually run at home.
Granted, you'll need beefy GPU power, because running this on the Spark will be very slow.
Today we release Ripple-65B Preview, our 65B dense LLM we pretrained from scratch.
Ripple sets a new state-of-the-art for U.S. open-weight models in agentic coding tasks, surpassing significantly larger models including Muse Spark Open-816B, Beam-501B, Inkling-975B, and Nemotron 3 Ultra-550B.
Learn more: ricursive.com/ripple
Another good one incoming from the Home Assistant team!
All will be revealed next Wednesday! 👀
Join us for the livestream and set a reminder so you don't miss the big reveal.
🔗youtube.com/watch?v=YpcrIy5f…
Home Assistant changed the home automation game and HACS changed Home Assistant for the better.
Now HACS is part of Home Assistant. Excellent work!
Home Assistant 2026.11! 🎉
HACS is now the built-in Marketplace 🛍️, exclude items from targets 🧠, a new way entities get their names 🏷️, LoRaWAN 📡, and 38 new integrations!
home-assistant.io/blog/2026/…
This is the case nobody puts in benchmarks: no internet.
A cloud agent can't fix the thing that knocked you offline, but a local one can.
Every home lab needs one model that works with the internet cable unplugged.👍
MoE models bigger than your GPU, 83 tok/s on a single 3090, written in Rust down to the CUDA kernels.
Qwen Flash is one of my daily drivers, so this goes straight on the test list.
The "you need a datacenter" crowd is running out of arguments.
bloomery
Runs MoE models bigger than your GPU, 83.36 tok/s for Qwen3.8-Flash-Next on one RTX 3090
It is an inference server for mixture-of-experts models, written in Rust from the HTTP server down to the CUDA kernels.
Experts are split between the card, host RAM and NVMe, and the ones called most move onto the card while it runs.
On one RTX 3090 24 GB, Qwen3.8-Flash-Next decodes at 83.36 tok/s and MiniMax-M3.1-Flash at 23.41 tok/s, both with part of the model on the CPU.
Two cards with 72 GB between them serve Muse Spark Open 816B-A42B, 225.6 GB of weights.
It works as a drop-in llama-server with the same OpenAI and Anthropic API and GGUF files.
NVIDIA cards, the DGX Spark and Apple Silicon through Metal are supported, and prompt caches survive other requests.
github.com/midagedev/bloomer…
Tencent's Hy4 is out and looks better than Preview: 770B MoE, vision, computer use, sold as "among the leading open models."
Weights? Not out yet. Arena lists it as proprietary for now.
Appreciate the work, but "open" is a download link, not a press release.
Codex users on HN report the "Improve the model for everyone" switch quietly turning itself back on after they disabled it.
If you have to check a privacy setting every week, you don't control it.
My local model has no switch, it doesn't need one.
Here's a video of local AI folks going to Qwen4 immediately after release.
Qwen just leaked a 120B model that can see, hear and think...
"qwen4-omni-120b-think" showed up in the test files of Qwen's own code, along with a 1M token context window for Qwen4
and that's on top of the lineup we're reportedly getting:
> Qwen4-Max
> Qwen4-Plus
> Qwen4-Flash
> Qwen4-27B
Qwen's Omni models handle text, images, audio and video all in one model, so a 120B one with reasoning would be huge
Alibaba Cloud's CTO reportedly said "next week" on stage last Wednesday, so this could all drop any day
Qwen4-27B is going on my 3090 the second it does
also I found a video of us all migrating to Qwen4-27B:
The next big battle in AI model development will be who can their hands on as much private, proprietary data as possible.
That will allow models to gain capabilities which their competitors can't touch.
If you enjoyed the preview, here's the real deal!
👋Hy4 is here, new with vision.
Firmly among the leading open models.
More productive everywhere: finance, frontend, game, officework, research…
Built responsibly, with cyber/biosecurity in mind.
Try it out and let us know!
🔗：hy.tencent.ai/research/hy4?l…
People will download a 210GB game without blinking, then call a local AI model "too big."
My daily driver is about 119GB and it actually does work for me.
You have the disk. You have the RAM and probably a GPU. You just haven't tried yet.
Revolut moved its engineers to open-weight GLM models running on GPUs it rents itself, at about a tenth of the cost and with no usage caps.
Frontier models are now just for edge cases.
The banks figured out what us local AI bros already knew😂
BREAKING: Revolut moved ~2,000 engineers onto self-hosted GLM for coding and pushed Claude Code and Codex to the exception list.
Per David Abitbol, who leads AI ops there: 56 Nvidia B200s, a blended 30 cents per million tokens, roughly a tenth of what they paid Anthropic and OpenAI. Engineering lead time improved 70%.
Here's the part that matters: the gains showed up when tokens stopped being rationed. Engineers quit saving AI for the hard tickets and pointed it at everything.
Frontier models become the exception, open weights become the default. That's a pricing problem for every lab heading to IPO.
h/t @OliverMolander
Would you move your team off frontier APIs if the bill dropped 90%?
Claude Duet is still a US-only waitlist.
The open-source version, Open Duet, already runs a team of agents with memory, a lead agent splitting the work, and asking before it acts, all on your hardware.
Works with Ollama or whatever you choose and you can put it to work tonight!
Read this one carefully: 1,035 tok/s is the TOTAL across 64 sessions. One chat gets ~127.
Both matter if you run agents at home:
- single speed = how fast one agent thinks
- batched speed = how many agents one GPU can feed
I've got only two sharing one Qwen 27B, so many to go!
A single RTX 3090 is reportedly pushing 1,035 tokens/s of aggregate throughput while serving 64 concurrent Qwen3.8-27B sessions.
The interesting part? This is a 24GB GPU running with a 250W power limit.
No multi-GPU setup. No expensive inference accelerators. Just one RTX 3090, vLLM and an open-source project called HyperQwen.
The reported configuration
• GPU: NVIDIA RTX 3090 (24GB VRAM)
• Power limit: 250W
• Model: Qwen3.8-27B
• Inference engine: vLLM
• Single-session generation: Approximately 127 tokens/s
• 64 concurrent sessions: Approximately 1,035 tokens/s combined
• Longest tested context: Up to 150K tokens
There are two numbers worth paying attention to.
The 127 tokens/s figure represents generation speed for a single session.
The 1,035 tokens/s figure is the combined output across 64 simultaneous sessions.
In other words, one conversation is not receiving 1,035 tokens every second. The server is distributing available GPU resources across multiple requests and accumulating the output from all of them.
That distinction changes how we should look at this result.
Local inference is also about serving multiple users
Most local LLM benchmarks focus on one question: How quickly can my GPU generate a response?
But developers building AI applications have another problem to solve.
What happens when several agents and applications need the same model at once?
Imagine one machine running:
• A coding agent modifying a repository.
• A research agent gathering information.
• A document assistant processing files.
• A code reviewer checking generated changes.
• Other applications sending requests to the same model.
Running a separate model instance for every application would be wasteful.
A shared inference server allows these applications to connect to one endpoint and use the same model infrastructure.
That’s the area HyperQwen is exploring.
Why more concurrent sessions increase throughput
A GPU serving one request may have processing capacity that isn’t fully utilised throughout generation.
With multiple requests arriving, vLLM can schedule their execution together through continuous batching.
Instead of treating every conversation as an isolated workload, the server combines work from multiple requests to keep the GPU busier.
Efficient KV-cache management also helps the system handle concurrent sessions without wasting as much memory on attention state.
There is a trade-off, though.
Higher aggregate throughput doesn’t automatically mean faster responses for each individual user. As more sessions compete for compute and memory, per-session generation speed and latency can change.
The objective is to increase the total amount of work the server completes, not necessarily make every conversation faster.
Getting started
HyperQwen provides two Docker Compose profiles:
Single-session mode
docker compose --profile single
Batch mode
docker compose --profile batch
The initial setup also requires some preparation. The first launch reportedly downloads around 9.5GB of Docker images, followed by a local model requantisation process involving approximately 20GB of data.
That initial work needs to finish before running the inference benchmarks.
Project: HyperQwen
GitHub: syv-ai/HyperQwen
Why this matters for developers
GPU specifications alone don’t tell us how useful a machine will be for running AI applications.
Single-session generation speed measures how quickly one user receives tokens. Aggregate throughput measures how much output the server can deliver across multiple users and applications.
Both are important, but they describe different aspects of inference performance.
HyperQwen’s reported results show what can be achieved by optimising the serving stack around a relatively affordable, widely available GPU.
For developers building local AI services, shared inference environments or multi-agent systems, this is worth exploring.
Big step forward in personal manufacturing.
Every time Bambu Lab release something, you know it's gonna` be good!
Bambu Lab H3C launches today.
Nine independent PrintTools™ that allow multiple nozzle sizes in a single print.
Powered by SwapTek™, Print-to-print tool switch only takes 6 seconds.
Multiple toolheads with multiple possibilities. Print, cut, draw and deboss. All on the same machine.
We built this machine. Now we want to see what you make with it.
--------------------------
To celebrate the launch, we're sending one H3C Combo to a maker ready for it.
How to enter:
• Follow @BambulabGlobal
• Like + Repost with: "Bambu Lab H3C. Tool Changer. Game Changer!"
• Comment: Which H3C feature impressed you the most?
Winner revealed: Nov 6, 4AM EST
Explore the H3C: share.bambulab.com/4yoyNdz