dippo

My old Echo Dot run Hermes now, instead of Alexa. I built a Home Assistant integration that connects it to a Hermes profile, with its memory and tools. Code, setup skill and the hardware recovery notes are public. github.com/majorminorlabs/he… @NousResearch

Building my own personal Grok Bot hardware, still work in progress 👀

Testerday: the Echo Dot Today: the Echo Show 5. With a screen. It shows what it heard, writes the answer as it speaks, and shows what it's working on. And the screen is just a web page so it can be anything. @NousResearch

Testerday: the Echo Dot Today: the Echo Show 5. With a screen. It shows what it heard, writes the answer as it speaks, and shows what it's working on. And the screen is just a web page so it can be anything. @NousResearch

First step. Successfully integrated @NousResearch Hermes into the unit working touch and response.

My old Echo Dot run Hermes now, instead of Alexa. I built a Home Assistant integration that connects it to a Hermes profile, with its memory and tools. Code, setup skill and the hardware recovery notes are public. github.com/majorminorlabs/he… @NousResearch

Qwen3.8-27B reportedly went from 82 to 381 tokens/s on a single RTX 3090 with 24GB VRAM. That’s roughly 4.6× the baseline throughput, using the same GPU, model and inference engine. Developer TeksEdge achieved this by combining several inference optimisations rather than relying on one setting. The progression is particularly interesting: Stock baseline: 82 tokens/s Optimised MTP: 114 tokens/s DFlash2 + context lookup: 138 tokens/s Optimised lookup verification: 381 tokens/s What is making it faster? 1. Multi-Token Prediction (MTP) Normal autoregressive decoding generates one token at a time. MTP introduces a way to propose multiple future tokens and verify them together. When the main model accepts several proposed tokens, it avoids some of the sequential decoding steps that would otherwise be required. 2. DFlash2 DFlash2 provides another speculative decoding mechanism. It uses a lightweight draft process to propose tokens that the main model can verify. The benefit depends on how many proposed tokens are accepted and how efficiently verification is performed. 3. Context lookup This is where the experiment gets particularly interesting. Some workloads contain text that already exists in the input. Reproducing a long document or transforming code with repeated sections can involve generating sequences the model has effectively seen before. Context lookup searches the available context for matching sequences and proposes them for verification instead of generating everything from scratch. TeksEdge reports using blocks of up to 16 tokens, with approximately 15 accepted per block. In a 25,000-token document reproduction test, throughput reportedly increased from around 260 to 382 tokens/s. That is a workload especially suited to context reuse. Ordinary chat, where much of the output is new, may see smaller gains. 4. Prefix caching and faster verification Coding agents often resend identical system instructions, tool definitions and repository context. Prefix caching allows the inference engine to reuse computations from previously processed prompt sections rather than processing the same content again. TeksEdge reports reducing time to first token from 22.4 seconds to just 0.56 seconds, approximately a 40× reduction. The important distinction The 381 tokens/s figure is not the expected speed for every conversation. TeksEdge also reports around 133 tokens/s for regular chat. The higher result comes from a workload that benefits heavily from context lookup and optimised verification. So, the result needs to be understood across different workloads: Regular chat generation Long-document reproduction Prompt processing Context reuse Speculative token acceptance These are not interchangeable measurements. Why this matters for local AI The setup remained: GPU: RTX 3090, 24GB VRAM Model: Qwen3.8-27B Inference engine: llama.cpp The performance gains came from reducing repeated computation and making token verification more efficient. For people running local coding agents, this is worth paying attention to. A model’s practical speed depends on more than its parameter count or the GPU running it. The inference pipeline can make a major difference, especially when workloads contain reusable context. Sometimes, getting more performance from a 24GB GPU means making it do less repeated work.

A brutal year for Bitcoin is just getting started. Give me three minutes to explain why you should dump it now. Outflows from ETFs are signal and will continue. We will test the 57k low again in the next 12 months. Why? Because there are far too many dynamic places to put your money that are creating MUCH more value. Why put your resources into a dead network that offers nothing beyond a store of value? What's the point of place the BTC be if Bitcoin's Ponzi-scheme nature has gone away!?! The viral nature of bitcoin was it's power... if you didn't own you were MOCKED! Now? Completely reasonable, heck, SMARTER, to ignore bitcoin and place other bets! No one is trying to convince people to buy Bitcoin anymore... the memes and NFTs are gone. No one is saying $ BTC can change the world, because it won't — and BECAUSE AI HAS! Just sell and shift from Bitcoin to Super Intelligence because one provides little to no value, and the other provides unlimited upside for humanity. This should be obvious to everyone right now. Bitcoin will never break its all-time high again.

Ornith-1.5-122B on one RTX 3090. Experts in system RAM via --n-cpu-moe, plain llama.cpp, cloud pod. Q4 at 32K: 548 tok/s prefill, 15.5 tok/s decode Q3 at 32K: 311 tok/s prefill, 26.3 tok/s decode Q4 perplexity +0.21% vs BF16. $1.53 total. @ornith_ @UnslothAI