Dominic Cabansay

am i the only one that just found out that claude has a voice function where claude talks to you?

i've been trying to go deep on async rl recently, this @huggingface blog post: is such an insightful read. on a high level, 1. trl's async grpo trainer can now train just a LoRA adapter and only send that adapter to the inference servers like vLLM, not the whole model. 2. normally you'd need a fast direct gpu-to-gpu connection to ship GBs of weights after every update. but a rank-1 adapter is basically a few MBs, so they skip all that and use a shared storage bucket instead. i feel like the easiest way to think about it is basically a versioned deploy. the trainer saves each new adapter (v1, v2, v3...) to a shared bucket, then just tells every server 'hey, load v7 from this path.' and the servers never get the weights sent to them directly, just a pointer to where they are, so the trainer and the servers can live on totally separate machines 3. there's also a small proxy that sits between the trainer and the vllm servers, so the trainer thinks it's talking to just one server. it sends each rollout to whichever server already has that prompt cached, and makes sure every new adapter reaches all of them the best part of reading this blog was seeing the results. just by reading their metrics and tuning the run, they cut the time to run 500 steps of rl from 3h27m down to 53 min, with basically the same reward definitely worth a read if you're doing rl on a small setup, as someone who can't fund thousands on a full async rl setup, this is definitely worth a try!