Product Manager & AI Engineer working on Gen AI & ML. Opinions are my own, not my employer's. RT !=endorsement
Seattle, WA
Joined November 2021
- Tweets28.9K
- Following7.5K
- Followers4.1K
- Likes112K
Hey @_samirism My dot seems to have a bug in the most recent iOS release, if i switch speakerphone on, it immediately turns off. @OpenAIDevs
Okay, did @OpenAI change the $200 plan again or something? Legit chewed through 100% of the weekly quota in 6 hours yesterday and 22% of another in another in another 2? @OpenAIDevs @thsottiaux
iamrobotbear (bk) retweeted
Why don't we at TypeSafe call Jev a decision model?
If you think of Jev's questions as making *decisions*, I believe you'll use it at the wrong layer of abstraction and your reliability will suffer.
I wrote all about this on our site 🧵👇
iamrobotbear (bk) retweeted
18 of the 20 best open models today are Mixture-of-Experts. Dense is basically over.
And yet the know-how to train a good MoE still lives mostly in the heads of a handful of teams.
So we spent 800k GPU hours running the experiments ourselves and wrote everything down. Super happy to finally release "The Ultimate MoE Training Handbook", free and open, organized around the questions training teams hit:
- how many experts, how big, shared or not?
- why is half my step spent in all-to-all, and how do I hide it?
- why is my run 25x slower with no error in the logs?
- softmax, sigmoid or sqrt(softplus) router, and why keep it in fp32?
- aux loss or DeepSeek's loss-free bias?
- upcycle a dense checkpoint or train from scratch?
- can I serve with fewer experts per token than I trained with?
⚠️ Warning: it's long (~70k words, budget a few days, sorry) but with 55 interactive figures, code you can run and numbers from our own runs.
Read it here: huggingface.co/spaces/huggin…
iamrobotbear (bk) retweeted
We've gotten used to crazy progress every month in the world of frontier LLMs. And more recently, mid-sized, open weights LLMs (~20B - 200B parameters) hit that same fast-progress, upwards curve.
But the new, new thing is super fast progress in small, specialized models.
@JonPTaylor is training new models every week, now, for things like detecting whether speech is sarcastic. These models can take text input, audio input, or both. They can be small enough to run on CPU in <50ms. You can combine them with LLMs, decision models like Jev, other classifier models, etc.
Specialized model training/fine-tuning is one of the things that coding agents make it possible to do 100x faster. So go make your consumer GPU go brrrrrrrr and train some tiny models.
New week, new tiny model for Pipecat.
Pipecat Voice Reader listens to how something is said: tone of voice, energy, laughter, speaking rate and potential sarcasm.
It's a fine-tuned version of LAION’s affect-trained Whisper encoder, in two sizes (base or tiny), and runs on CPU in well under 50 ms per turn. In a Pipecat pipeline it runs beside the text tone reader, which reads what was said in the context of the previous exchange, providing broader context to the harness and metering.
This is one area where a cascaded pipeline has a real edge over speech-to-speech... each turn gets explicit, calibrated readings of both the words and the delivery, with context of the single turn and the entire conversation.
Available in two weights (base and tiny): huggingface.co/pipecat-ai/vo…
iamrobotbear (bk) retweeted
Software can now understand human language and act on it, in real time.
Introducing Sage2
- Extract: natural language in, structured data out. 166x cheaper than Opus 5.5, beats @Cloudflare's Clairic
- Decide: the first decision model with streaming. 3x faster, because it reads the context as it grows.
Sage2 is a new primitive for developers. Get 20M tokens free 👇
Also available on @near_ai and, for agents, via @mpp with @Stripe cards and @Tempo stables.
iamrobotbear (bk) retweeted
As a dev, I am bullish on this new type of system-one extraction model.
It's great for evidence-based extraction and perfectly compliment decision models like Jev.
With Sage 2, you describe the fields you need, and it returns structured values with the exact source text and a probability for each result.
It seems to outperform Cloudflare's Clairic, one of the stronger extraction models I have tried.
We've been experimenting with extraction models, and I'm excited about applications like processing and analyzing research papers.
Worth checking out.
Software can now understand human language and act on it, in real time.
Introducing Sage2
- Extract: natural language in, structured data out. 166x cheaper than Opus 5.5, beats @Cloudflare's Clairic
- Decide: the first decision model with streaming. 3x faster, because it reads the context as it grows.
Sage2 is a new primitive for developers. Get 20M tokens free 👇
Also available on @near_ai and, for agents, via @mpp with @Stripe cards and @Tempo stables.
iamrobotbear (bk) retweeted
@treetowntree and i created neuron lens: a database of every single neuron and attention head in qwen 3 8b and gemma 4 12b read using the R-lens (a J-lens variant) and what we call the "S-lens". you can find it at neuron-lens.com/, along with the tools we used to make it.
using the R-lens to read neurons' input and output projections turns out to be very effective at determining what a neuron does. we're able to legibly read what a majority of neurons do when fired: the lens qualitatively predicts the result of steering over 90% of the time!
the same trick lets us read attention heads' QK circuits and OV circuits, showing us both what they attend to and what they do with it. however, the R-lens has two important flaws: its accuracy degrades in early layers, and it can't separate the MLP block and the attention block.
we found ways around both these issues. we first modified the r-lens by fitting it on input tokens rather than output tokens to get the s-lens, which is accurate in early layers but inaccurate in late layers. we then fitted R- and S-lenses to the activations after attention but before the MLP to get "mid-lenses", which are much better at deciphering MLP inputs and attention outputs. using both of these techniques, we were able to interpret most of the models' layers.
iamrobotbear (bk) retweeted
a lot of decision models came out in the last two months. almost all of them work the same way. they read text, wait until the whole input has arrived, and then give you an answer
that works for a support ticket. it doesn’t work for a robot, a phone call, or a video feed, where the input never really ends and the decision has to happen while it is still coming in
so we spent half of our YC money training a model:) RightNow takes text, PDFs, images, voice, code, robot state and video, all in one model, and you can stream up to 1M tokens of context into it. it updates its decision as each piece arrives. on text it decides in 30 ms and we will make it less with our kernel optimization
we trained it with what we call Reflex Training. the current decision models read everything first and answer once. RightNow is trained to decide on whatever has arrived so far, to hold back until the evidence is there, and then commit. it serves on what we call Reflex Architecture which keeps a probability for every option and updates it as each part of the input comes in
we call this kind of model “System Zero”. system 1 is fast thinking and system 2 is slow reasoning. system zero is the step that comes before both, which is just deciding
we will publish a technical paper on Reflex Training and Reflex Architecture. the model itself will stay closed source and available via RightNow API
this is our first checkpoint and it’s still a little buggy. we launched it early because the fastest way to make it good is to have people use it:)
RightNow is out today. our first “System Zero” model
it streams in text, JSON, PDFs, images, voice, codebases, robot state and video, and returns a decision with a probability for every option while the input is still arriving
30 ms decisions on text. up to 1,024 questions in one call. output is free
system 1 is fast thinking. system 2 is slow reasoning. “System Zero” is the step before both -- the decision itself
this is our first checkpoint. there is a lot of work ahead, and we decided to ship early to get your feedback. more features and fixes are shipping this week, with more checkpoints after
rightnowai.co
iamrobotbear (bk) retweeted
Introducing *Switch*: a *harness framework* for *decision models* like Jev
I looked for this the other day and was surprised to not find anything like it
You obviously cannot use a decision model like an LLM in a coding agent harness like @pidotdev, because Jev cannot generate tool calls
But what if there was a framework that "generated the tool calls" for it programmatically, and Jev only had to pick out of that finite set of choices
That is what switch does. You pick a "domain" to implement the decision model in, and it provides a core API to define your state, and the functions that will derive decision model questions out of that state. Chess is a domain, as well as being a rubric judge for evals
It basically standardizes what I imagine a lot of people are already vibe-coding, to an abstract interface: calling a decision-model API through multiple steps, i.e. an agent loop over finite choices
Because of the finiteness of the output of a decision model, you cannot have a general pure decision model harness. Instead, you need to implement specific ones that generate the choices programmatically (or have an LLM generate the choices, but I find the former more interesting)
A primary use case I have for this is use as a judge in an eval. For example, At the end of a task run that involves a bunch of documents, most people currently follow this approach: extract text out of PDF/docx/md files, and dump them right into the context
But what if the visual formatting of the output is a judging criterion? Then you have to dump an image of every page in the input state. But what if you have 200 pages, and your question does not need looking at all of them? What if Jev could navigate through the haystack and find the info it needs?
I have noticed that Jev is not so good at chess nor being a Decision-Model-as-a-Judge. It sucks at chess, and it does not perform well enough to replace GPT 6.1 Sol as a judge in the domain I am working for
But switch is not only for Jev! It is for any decision model including the ones supported by the pi-ai package
So once you can generate enough data with a better model, you should be able to pick an open source one and fine-tune it on your use case to perform better! And the model should be trained in a harness like switch's to be especially performant and cost-efficient
I wanted to get this out because I find the idea very exciting, at least for benchmarking. Jev is currently not there, but decision models obviously will be there at some point, and we can take them there! There are also a lot of potential gains from picking a specific domain, and creating a looping agent that does a job for 100x cheaper!
Let me know if you have any ideas, I am quite keen to try this on new domains!
Repo: github.com/osolmaz/switch
iamrobotbear (bk) retweeted
Meet Pika H3 Flex, debuting today as the #1 open weight model on DesignArena’s t2v board.
It’s built for professional iteration, with exportable audio stems, resolutions up to 4k, and faster-than-realtime generation. It can produce a 15-second clip with soundtrack in as little as 7 seconds.
So, yeah. It’s kind of a flex 💅
iamrobotbear (bk) retweeted
Starting today, eligible ChatGPT Pro and Business subscribers can claim up to $500 per month in API credits to build apps and agents.
iamrobotbear (bk) retweeted
Meet hunch! Jev-like decisions in ~20ms without even needing a GPU, learning live from a teacher model.
Built on @GoogleGemma's EmbeddingGemma 2, it can works with any teacher model like @CloudFlare's Clef.
Multimodal, w/a systemone endpoint, slots into your existing stack 🧵
iamrobotbear (bk) retweeted
Introducing Vela 3.0 📡 Name it. Vela finds it.
Six open models (0.3B–27B): decisions + the spans you name. One call.
⚡ 380M: #1 Decision Index ≤500M
🎯 Beats GPT-5-mini at zero-shot classification
🔍 18 ms per span question
🤗 huggingface.co/collections/v…
#Jev #Decison
iamrobotbear (bk) retweeted
Watch this, understand it, and embrace what the future of reality will look like
Introducing Reality-Aware Rendering in @Simulon, a completely new way to create with the real world.
Place neural assets with your phone and act out the scene. When you stop recording, Simulon renders the take, reading everything in it (what you do, what you say and the place around you), and your assets respond to all of it.
The final is regenerated from your recording: a new performance of the scene, not a copy of the footage.
Available now on iPhone. Coming to Android soon.
iamrobotbear (bk) retweeted
When I said hunch could learn in real-time I wasn't kidding. It's like teaching a kid!
Behind the scenes, @Cloudflare’s Clef is helping refine the local model so it learns in real time, while @googlegemma’s EmbeddingGemma2 runs live on the CPU. 🧵
It's been a while since I showed any Jev stuff but I've been cooking on a way to use @GoogleDeepMind's EmbeddingGemma2 as an insanely fast real-time /v1/systemone endpoint (20ms on text, 100ms on images).
It's trainable in real-time, can distill an upstream teacher model, and is multi-modal by default. Hard to believe a @googlegemma model of this size can do so well with the right harness!
iamrobotbear (bk) retweeted
Introducing 𝙻𝙵𝙼𝟹: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence.
> 6 model instances, across text, audio, vision, and omni modalities, generative and decision models.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱: text, 256k context window.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱: vision-language model with video-input support for spatial understanding
> 𝙻𝙵𝙼𝟹-𝙰𝚞𝚍𝚒𝚘-𝟸𝙱: tool calls, voice chat, reasoning in 18 languages.
> 𝙻𝙵𝙼𝟹-𝙾𝚖𝚗𝚒-𝟸𝙱: an omni model combines vision, audio, and language.
> 𝚍𝟷-𝟸𝙱 and 𝚍𝟷-𝙾𝚖𝚗𝚒-𝟸𝙱: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input.
> fine-tune on 1 GPU.
> a new tri-hybrid architecture
> architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs.
> 𝙻𝙵𝙼𝟹-𝚅𝙻-𝟸𝙱 is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters.
> 𝙻𝙵𝙼𝟹-𝟷.𝟽𝙱 is #1 on the Pareto frontier of device intelligence 𝚙𝚒𝚙𝚎𝚝𝚝𝚎.𝚕𝚒𝚚𝚞𝚒𝚍.𝚊𝚒
Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device.
1/ 🧵
iamrobotbear (bk) retweeted
Genie 4 is quite an insane experience, generating game-like enviornments as you go. It isn't flawless but I can type "Neogothic brutalist city inhabited by robots and horses" & it creates one in a minute. Then I asked it to add dragons. Then a friendly otter NPC. All in realtime
iamrobotbear (bk) retweeted
Now that the dust has settled with Jev, here's where it works well and where it falls short
iamrobotbear (bk) retweeted
Introducing Reinforcement Learning from Collective Rewards (RLCR).
RLCR is a recipe that trains language models as agent organisations, which share a message board and workspace. We use it to train Avalon-M1-Mini (120B) and study its behaviour.
We find a number of interesting results, including:
🛠️ Learnt organisational hierarchies and divisions of labour
🕵️ Increased collective verification, through hires of verifiers and testers
👥 The rise of collective thinking, where agents think more about other agents and how to help them
We believe RLCR can provide a laboratory for studying collective intelligence and multi-agent safety.
Blog post and technical report 👇