selogreen

The way frontier labs have restricted access to cyber capabilities makes no sense. If you're considered a "defender" you're not allowed access to the "offensive" versions of the models (the ones that can do development/validation of malware and exploits). So for example, if you want to build some sample malware to test a detection system, nope. Or if there's a major crisis due to a critical vulnerability in PHP and you need to rapidly triage fake exploits to see if there are any real ones floating around so you can warn people, nope. Want to build a remote code exploit to make sure the detection rules you plan on proposing to the clients actually work? Hard nope. You gotta call your red team friend and wait for them to do your job for you, because "defender" are only allowed to play in the sandbox with the children. Not an expert, but I think that the ability to triage in-the-wild exploits during a major crisis affecting 40 million systems is possible maybe important. And then they want to ban abliterated open weight models, which half of the people are only using because frontier labs will only give them access to lobotomized models.

With LLMs, we like to think of "smart" and "dumb" as one axis (because we think of humans this way). I'd like to argue against this framing. Instead, try to think of "smart" and "dumb" as two different axes. A model can be incredibly smart AND dumb at the same time. For a good example of this, look at a Gemini model. They are incredibly smart: you can bench them and see the capability. The amount of knowledge Google bakes into their models is incredible. Yet when you ask it to do work, the amount of stupid things the models will do is similarly incredible. They are "smart" and "dumb" at the same time. Similarly, look at a model like Fable 5.1. It is not quite as smart as Astra, but it is significantly less dumb. Astra is still the smartest model available today, but it is also one of the dumbest, regularly doing things that make literally no sense whatsoever. Finding the right balance of both "smart" and "dumb" is a challenge that everyone needs to figure out, from users to researchers. If we stop thinking of it as either-or and instead realize both can exist together, understanding model behavior gets much easier.