Ed 🍀 retweeted
The servers we returned to service can become unresponsive again once they receive traffic; we're analyzing diagnostic data to determine what's triggering the behavior. In parallel, we're temporarily limiting which types of traffic can reach the affected infrastructure in some locations. Additionally, we're rebuilding a subset of affected servers using a previous configuration to validate if this prevents them from becoming unresponsive. More info under EX1550543 and MO1550821 in the Microsoft 365 admin center, as well as status.cloud.microsoft if the admin center is not available to you.
Ed 🍀 retweeted
HOLY: Anthropic's Claude Code engineer Addy Osmani is warning about a brutal AI paradox: the more coding you hand to agents, the more you risk losing the very skills needed to supervise them.
He calls it "cognitive surrender." AI-generated code looks pristine, tests pass, and engineers start approving work they no longer understand.
Anthropic's own research warns: "supervising Claude requires the very coding skills that may atrophy from AI overuse."
Osmani makes a striking observation: "When you're producing more than you can stand behind, you're no longer reviewing. You're hoping."
Ed 🍀 retweeted
Local AI is getting better 🙃🙃
What you’re listening to isn’t a real person. It’s AI-generated speech, running entirely offline.
I’ve been building my own voice processing engine — think of it as my homemade ElevenLabs — using the @qvac SDK and Alibaba’s CosyVoice.
High-quality voice generation, running locally, with zero API costs.
This is just one part of the all-in-one AI voice processing desktop app I’m building
New updates coming soon + the full codebase
Ed 🍀 retweeted
Fun fact: the creator of CoreAnimation is one of the creators of SwiftUI!
The decision to run animations in the app process rather than the render server (as CoreAnimation does) was one of the earliest and most fundamental decisions we made when designing SwiftUI.
Some of the most distinctive experiences of using an iPhone, like UIScrollView rubber banding and SpringBoard app switching are only possible as client-side animations.
But these experiences aren’t easy to build with UIKit!
This is because if you want a UI to be fully interactive and interruptible by the user while it’s animating, it has to be running in the same process as the event handling.
In fact, in iOS 18 we added new APIs to AppKit and UIKit to adopt the SwiftUI client-side animation model.
There are a lot of valid critiques of SwiftUI. The fact that SwiftUI doesn’t use CoreAnimation for its animations is not one of them.
SwiftUI isn’t native and here’s why SwiftUI apps may sometimes feel janky. It ditches the key element that made animations run so smoothly on iOS since the original iPhone: Core Animation.
In SwiftUI animations run in-app without CA backing. Block the main thread: SwiftUI ball stalls while UIKit’s keeps moving on the render server.
Ed 🍀 retweeted
TIL SQLite converts integers to text two digits at a time
converting an integer to text usually means a loop of /10 and %10
sqlite avoids most of those divisions by keeping a 200-byte string of every two-digit pair from 00 to 99, and pulling two digits at a time out of it by direct index instead of computing them
Ed 🍀 retweeted
“b-b-but startups need to be in person!”
nope, zero to multibillion dollar sale in 3 years with fully remote workforce
what you ACTUALLY need is top talent obsessed with solving the customer problem
OpenRouter is joining Stripe.
We started OpenRouter with a simple mission: intelligence should be multi-model. Today, we are the largest AI marketplace & gateway, processing 10T+ tokens daily on 400+ models.
Joining @Stripe gives us the opportunity to accelerate that mission.
Ed 🍀 retweeted
This is the McLaren McL 6GT: The first manual McLaren in over 30 years.
An homage to the MG6GT owned by founder Bruce McLaren, the McL 6GT will be a light, analog-based super car built in low numbers.
Announcing Discovery Loop!
I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.
♾
Learn more at: discoveryloop.com
Ed 🍀 retweeted
Releasing the model weights and technical report of Kimi K3.
Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window.
New model architecture: 2.5x the intelligence per unit of compute, not just more params.
Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale.
Model weights: huggingface.co/moonshotai/Ki…
Tech report: github.com/MoonshotAI/Kimi-K…
Tech blog: kimi.com/blog/kimi-k3
Ed 🍀 retweeted
Remember it went from
- It can hack everything
- Cybersecurity is at risk
- It is too dangerous to release
- People will make bioweapons
- It will literally cause world war
to
"ClaUde faBlE 5 wiLL be IncLudeD iN All maX anD tEaM pReMiUm pLaNs"
Beginning July 20, Claude Fable 5 will be included in all Max and Team Premium plans, at 50% of limits.
Pro and Team Standard users will continue to have access to Fable via usage credits, and will receive a one-time $100 credit.
Demand for Fable has been challenging to predict, which is why we rolled it out to subscription plans in stages, extending access several times as we secured additional capacity.
Ed 🍀 retweeted
I agree with this post whole heartedly but I’d push it even further. The interconnect IS the binding constraint for AI even more so than memory. If we want faster inference & training with better economics we are best served by designing our interconnect first and then working backwards towards the optimal chip architecture.
Today’s chips weren’t really designed with this principle in mind. There is no better example than running autoregressive decode on a GPU. Despite all those reticle sized logic dies & CoWoS integration decode runs at under 20% of peak FLOPs on Blackwell, wasting silicon and burning power while waiting for memory. The naive solution has always been to increase memory bandwidth whether that’s adding more HBM or using SRAM. However, that is a vast simplification of the problem which I’ll explain later. But if you were clever you’d have realized while reading that you could feed those idle FLOPs by streaming weights over the interconnect itself. Wallah 🪄 you just discovered the idea that forms the basis behind disaggregated memory from first principles.
But sadly this currently doesn’t work on Nvidia’s hardware. NVLink5 carries 1.8 TB/s against 8 TB/s of local HBM, and scale out is 80x behind that. The “pipe” is smaller than the memory at the other end and thus leads to worst token/sec if its relied upon. But we get an interesting lemma out of this which is that remote memory is only as fast as the interconnect. Therefore you must balance the pipe for the memory it attaches to. SRAM needs an 80 TB/s link, HBM needs 2+ TB/s, and LPDDR gets away with a couple hundred GB/s. So Nvidia selling a rack of 72 GPUs, each GPU’s memory is pretty segregated. The core idea is still sound though but this raises a question, why would Nvidia build a fabric that’s high bandwidth and high latency leading to memory access being segregated per GPU?
It’s because they were optimizing for training over inference. Training is dominated by collectives on huge tensors, and a couple microseconds of latency on a huge all reduce operation is just noise so the bandwidth gains justify the latency tradeoff. But more importantly, this also works because it matches what the chip is good at. GPUs are great at hiding latency with occupancy (also what allows them to be OK for training) but bandwidth is the only thing warp switching can’t create. You can justify a 224G PAM4 + FEC with overhead when you have a chip that’s designed to be latency tolerant as well. It’s a latency tolerant fabric for a latency tolerant chip. Maybe a good design for training but inference inverts this completely.
Now everyone knows decode is bandwidth bound so you might assume again that more/higher BW memory and thus higher BW interconnects are necessary. However, it’s the exact opposite and the name of the game is actually lower latency and that’s why despite having high bandwidth memory MFU on decode is still so low and also why I made the point earlier that the interconnect is MORE important than the memory itself as well as the chip architecture. In part two I will explain why lower latency interconnects are not just ideal for inference but also allow you to get away with a smaller cheaper memory and a simpler chip architecture.
chips get all the love but the interconnects across all levels from c2c to rack-to-rack are as important, and many chip makers are sleeping on it until very recently. Even most interconnects people just want it to be as transparent as possible, just send and receive the bits with lower error rates and lower energy per bit, wrong long-term direction imo. Interconnects are part of the living creature, so many things happening in your blood vessels in addition to just moving stuff, and your axons do much more than carrying spikes.
People do not appreciate interconnects, smaller volume, poor margin, messy ecosystem, manual process, it's been a spiral of grinding, and it is largely invisible. How often do you see people tearing down transceivers and die shot of DSP chips vs logic chips? How often do you see high res pictures of all the connector's gold fingers on the NVL72 cartridge? Because it sounds boring, it's just making contacts, shoving electrons and photons, what a simple problem.
But that is deceiving, and theres so much to it. You might want 576 to begin with, had to cut down to 288, then to 144, and finally to 72, and that barely worked first time. You are entering the domain of analog and mixed signaling, you are fighting copper real estate with power delivery, you are getting impedance mismatch and reflections and interference at every stupid interfaces, your optical components' and connectors' backreflection is making your laser mode hop.. And we are not even going into the thermal and strain-stress, the reliability of how many times you can actually mate your connector, the horrible jobs people are doing across the stack from science-project-originated photonic PDKs to hand cleaved laser dies to optical engines to rack manufacturing, on spec-ing out the requirements, the tolerances.. On top of all these, people thinking about where the bits should be going and people who know what the bits have to go thru are two totally different groups of people.
But it is shifting, pluggable volumes shipped are doubling and tripling for scale-out, scale-up domain asks for much higher bw than scale-out, and interconnects are inevitable even if you cram as much compute and memory onto a single wafer. People will see it always has been interconnects, the chips people have already been doing it on the chips, that you can sort it out with your chip designers and foundries, and now you need to work with more people to sort it out from chips to boards/trays to racks to pods to data halls and data centers. These people speak very different languages and care about very different things, and it will take a lot of effort to pull order out of all the chaos.
At the end of the day, it is such a crazy problem to work on, such a beautiful thing to make, millions of amps of current flipping 1e20 of flops, sextillions of photons carrying thousands of terabits per second, a few tons of copper, tens of thousands of fibers totally few hundred kilometers, one scale-up domain. You absolutely need a group of people that appreciate the beauty and care about the craft behind the grinding to make it together.
Ed 🍀 retweeted
I helped debug early C compilers from the first I got on magtape from Jay Lepreau at Utah, the Portable C Compiler port to my university’s beloved DEC 36-bit mini. Yes, we had to read asm until bug count way below the hallucination rate of an LLM. Probabilistic genAI != Compiler.
i dont _fully_ agree with this BUT i do generally trust codex with rust (shout out rust compiler and tooling). i do tend to treat ai code gen + ai code review sort of like a compiler (although i still read every diff and review all the code, i also do this for C++ / assembly i write personally much of the time as well) and imagine this to increasingly be the case in the future. that said, i _still_ do not trust this process for c++ or typescript (for very different reasons) and python code tends to be a mess no matter who writes it so its treated as an exception
Ed 🍀 retweeted
Finally, my local chatbot can search the web, and all of this runs locally, no API key, just raw crawling and a model with of 700mb model weight.
Age inferencing is coming along nicely 😗.
@qvac
Ed 🍀 retweeted
Weekend experiments: got local TTS running on mobile with QVAC, and the results are looking really promising. It's great to see more AI workloads moving fully on-device.
New article dropping soon with a deeper dive. Mobile inferencing is coming along nicely. 🚀
@qvac













































