@modal and @tensarahq i like math and ml
hbm
Joined September 2019
- Tweets308
- Following352
- Followers515
- Likes709
Harmya retweeted
The secret to Modal's speed is our custom lazily loaded container filesystem, ImageFS. ImageFS is built on a massive tiered distributed filesystem, layering local NVMe disks, regional and p2p caches, and global blob stores.
When we launched VM Sandboxes this month, we wanted to make sure they could take advantage of ImageFS in the same way our gVisor containers do. To do that, we built out a new filesystem stack that tricks a VM's ext4 driver into thinking it's talking to a normal disk.
Harmya retweeted
The only part of this article I used AI for was the cover photo, I swear.
Link to session - chatgpt.com/share/6aa3732d-0… View article
Harmya retweeted
I sat down with @silasalberti, Head of Research @cognition, to chat about the intersection of RL and inference -- from training frontier coding models to running inference at scale.
Watch till the end for bonus doggo! :o
0:00 — Intros
1:04 — What's hardest to get right in an RL run
3:39 — How Cognition started using Modal
4:03 — What to weigh when buying inference
4:58 — The latency/throughput Pareto frontier
7:14 — DFlash reveal (!!)
8:44 — Why faster runtime matters
9:56 — Memory-bound vs. compute-bound
10:57 — Tree-based speculation
14:20 — Hot take: RL and inference optimization converging
15:28 — Online training of the speculator
16:22 — "Auto inference"
17:27 — Agents competing on optimization problems
18:27 — What inference providers still get wrong
Harmya retweeted
Modal has 460 TPS for Kimi K3 on release day!
Harmya retweeted
made a small artifact to explore update geometry across submissions to @kellerjordan0's nanoGPT optimization track
helped me build some intuition for what these optimizers do during training, maybe it will for you too!
Harmya retweeted
StableBrowse is a new browser layer for AI agents. They enable agents to navigate the web with 70% fewer tokens and 3-4x faster execution.
Congrats on the launch, @deepitshah5, @JayMehta446970, @_sarthak____, and @somanshshah!
stablebrowse.com
Harmya retweeted
Modal is proud to now support @claudeai Managed Agents with Modal Sandboxes.
Harmya retweeted
.@modal is hosting a mahjong night in SF ! expect lots of boba, chinese food/aux, and a good time
if you like poker but have never tried mahjong, this is your sign
partiful.com/e/MyzrBC93CaF2i…
Harmya retweeted
The future of artificial intelligence is physical.
@physical_int runs robotic control inference on Modal with >2x lower latency than the lag between your brain and your finger.
Harmya retweeted
help get Tensara a 6x6 inch spot in an SF mural 👀
paintastreet.com/#tile-019d5…
paint a street
submit a drawing that will go on the pavement of a san francisco alley
paintastreet.com
Harmya retweeted
ICYMI, there's a bit of a run on GPUs rn.
At Modal, we've toiled in the perf mines to make cold starts fast so that we and our users can make much better use of this scarce and dear hardware.
Come to our webinar next week to learn how we did it!
watch.getcontrast.io/registe…
Harmya retweeted
Robotics has spent decades optimizing for research. Deployment requires a completely different kind of person: operators, industrialists, and outsiders the field typically ignores.
There's a wave of people who want to build in robotics. The field doesn't know what to do with them.
New essay, Robotics Needs Fewer Roboticists* below 👇
Harmya retweeted
new page for the GPU Glossary! went under the hood on CUDA Graphs to figure out just why they are so hard to serialize
modal.com/gpu-glossary/host-…
Harmya retweeted
How far can balancing an update matrix’s norms take you?
@Abel_grg and I explored this by replacing Muon’s Newton-Schulz step with alternating row/col normalization. In our experiments, that alone is competitive with AdamW, despite no orthogonalization or second EMA tracking 🧵