I have found that a lot of eval signals for D1 models were actually just the distribution moving around and thus just having a better/worst default threshold behaviour, but the AUC were actually the same/very similar Tuning the temperature tries to get something more calibrated, but just tuning your threshold and looking at the behaviour of your model on your data goes a *very* long way Also we face the traditional false positive/false negative trade off we are facing for classification tasks for very long, so tune it for your needs We are very early in the decider model world, but just tuning the model for your case (threshold/prompt) can get you very large performance boost for free Ultimately we want something more robust, but get used to look at your data and properly eval the models, it will always be useful

In most of our benches, Decider models (not only Jev, not only ours), are very strong rerankers,often stronger than the dedicated rerankers Obviously there are very few 27B dedicated rerankers, but even the small ones are very good performance for their sizes

I mean, when you know the authors you already know it's going to be insane, but honestly the blog post was so interesting Stellar work, and very timely: the future is sparse, and getting strong sparse models easily with training SAE on top of dense/LI models is the way