Local AI. Building @picogpt Download: https://teapawt.net/t.co/o0PKhVM691 Discord: https://teapawt.net/t.co/KNirjAzl32
San Francisco, CA
Joined April 2008
- Tweets28.9K
- Following6.5K
- Followers6.9K
- Likes105K
Pinned Tweet
Apple Silicon + Gemma 4 fans: this is for you.
Pico AI Server now supports continuous batching with MLX-Swift.
43 tok/s on 1 stream.
26 tok/s per stream on 2 concurrent streams.
That’s 52 tok/s total. a 21% throughput gain on a six-year-old MacBook Pro M1 Max!
Good news for everyone with 32, 64, or 128 GB of memory. Im very curious how the small GLMs will stack up against Qwen
Ronald Mannak retweeted
The Ocean Cleanup is now $40M/year short of what’s needed to stop up to 1/3rd of all plastic flowing into the oceans by 2030.
That’s less than 1% of what America spends on Halloween decorations each year.
It’s sickening that we’re siding with people who hunt down civilians and even their pet dogs
This is so cool. An AI model that “speaks” Nintendo
I used Divyansh's approach to build an entire Nintendo into ~1GB of Qwen3.5 weights.
Qwenputer, meet Nesformer: github.com/dbreunig/nesforme…
I had early access to Skydive. It’s a cool, fun, and different way to interact with agent.
Your AI coworkers now have an office.
@getskydive is open to everyone. Build a team of agents, watch them work in a 3D office, and hand them tasks over Slack, email, iMessage or the web.
Claim your company’s domain to put your office on the map. They work for free this week.
These measurements are useful, but the conclusions need more context.
If the benchmarked setup duplicates attention and distributes whole experts between Macs, it could leave performance on the table. I’d love to see the same benchmark with attention split by head and each expert’s matrices split across both machines.
Additional bandwidth helped prefill more than decode in this test: four cables improved prefill by about 21% and decode by about 7% over one cable. However, the chosen setup doesn’t sync data as much as a full tensor parallelism setup, so more data is needed there too
Pico Server’s Qwen MoE implementation splits attention and each expert’s matrices. If you have two M5 Ultra Mac Studios and want to help benchmark an unreleased build with distributed inference, DM me.
TensorFold w/ 2 x Mac Studio M5 Ultra
TensorFold splits GLM-5.3-Flash at 4bit across two 256GB M5 Ultra Mac Studios over TB5 RDMA.
Decode rises from 82 to 109 tps.
On a 130K token prompt, prefill goes from 1,938 to 2,384 tps and first token latency drops from 69 to 56 seconds with four cables.
Only the MoE experts are split. Attention runs on both Macs, limiting scaling. With one cable, prefill stays near the single Mac result.
The biggest hack in *crypto* history? You’d think $5.7 BILLION would be the biggest hack ever. Period.
BREAKING: Binance has been hacked with wallets across the Ethereum, Bitcoin, and TRON networks seeing outflows of -$5.7 billion.
The hack includes $1.45 billion worth of Bitcoin and $875 million worth of ETH.
If confirmed, this would mark the biggest hack in crypto history.
We expect more information shortly.
Qty 10???? Follow the money people. It’s not that hard
(J/k coincidentally Eigenlabs will talk at our next AiOS meetup: luma.com/wy0z8tek )
Oh no. Captain Picard will always be my favorite captain
In lieu of flowers, memorial contributions may be made to:
Johns Hopkins Alzheimer’s Disease Research Center Fund
alzresearch.org/giving/
Alzheimer’s Foundation of America Caregiver Resources
alzfdn.org/donate/
Photo Credit: Tommy Garcia
Interesting
Finally figured out why the 512GB M5 Ultra costs so much.
The M5 Ultra has 64 RAM channels.
The 256GB model stacks standard 4GB LPDDR5X chips on each channel.
To hit 512GB, Apple needs custom, ultra-dense 8GB chips per channel.
That special RAM is why the price is higher.
Honestly, for regular buyers, getting two 256GB models makes way more sense and costs less.
Linking them with Thunderbolt 5 gives you 120Gb/s (around 15GB/s).
That connection is pretty slow on its own, but clustering two machines roughly doubles your total memory bandwidth.
Plus, you can just run two smaller models separately whenever you want.
I am probably just going to buy two 256GB units instead.
TL;DR: we haven’t scratched the surface what AI can do
So one of our ambassadors @divagr1925 has been working on something cool
He found a way to use Qwen's Gated DeltaNet state as memory, and built a 32-bit RISC-V CPU directly into the model's weights. So instead of relying on predicting the answer to a calculation, the model can execute instructions internally. It can also run C and MicroPython programs.
He calls it Qwenputer, love the name btw. The results so far are pretty impressive. On one arithmetic benchmark, accuracy jumped from 48.6% to 100%.
Pretty cool to see developers experimenting with Qwen's architecture. Great work, my friend!
Local AI software, libraries, and startups are booming. Within 3 to 6 months, many more people will realize local AI is already a viable alternative to the cloud.
Today we're coming out of stealth, introducing Neywa Labs 🚀
We built the infrastructure to build and deploy AI that runs on the devices people already own. Private, offline, unlimited.
One mission: make AI available to everyone in the world.
neywalabs.ai/