Barista | Looking for something to do | she/it
San Francisco, CA
Joined January 2023
- Tweets1.2K
- Following402
- Followers3.7K
- Likes9.2K
feel a bit depressed about the sf party scene at this point because everyone is always talking about the same tech topics. i was in seattle recently and i was surprised how much more fun it was to be there compared to sf
Alex Tomala retweeted
Today we are open-sourcing 🥕Karotte, our framework for building RL environments.
We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.
Now that there are a bunch of ai safety focused lab employees with a decent chunk of money, it seems to makes sense for some other people to pivot towards being personal assistants.
Helping manage their personal lives will probably make them better at their jobs and it exposes you to a lot of cool people :>
Someone sent me an email telling me that they finally managed to frame a letter I sent them, but I never recalled sending a letter and it isn’t in my handwriting 😳
Alex Tomala retweeted
Dario's essay points towards the right path forward. The details need working through, but the direction is correct for meeting this critical moment.
This is also why we recently put out our proposal for an industry-wide standards body for frontier AI. teapawt.net/demishassabis/status/2…
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
serious question
for people that focused less on data/benchmarks and more on architecture/infra, is there anywhere good outside of the major labs to work at?
yayayayayay
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.
OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.
I made a tweet earlier today about someone who had a good amount of money only paying for their share of the meal
Some clarifications
1) They mentioned they would pay 1/N (where N=7) and I did not state any concern or suggestion it would be net worth adjusted.
2) I didn’t confirm how much everyone should pay before the dinner
This part is pretty much on me and I do apologize for providing incorrect information. I should have never posted this tweet.
Some other people were asked if they are this person and while I wanted to keep this as an anonymized vent, it seems to have led to accusations that I’m uncomfortable with
i strongly endorse this statement
while i get the possible argument that “Anthropic must win at any cost” because of ethical concerns with competitors, my understanding of the OpenAI situation is that they are genuinely worried about how their latest models are behaving and are taking vital steps toward ensuring they are safe
this is one of the best case scenarios for Anthropic’s mission and i believe that they should demonstrate this by committing to a similar slowdown (or some sort of verifiable assurance that their models are safe)
other labs like Google DeepMind, TBD labs, and SpaceXAI should make similar commitments as well
this is a critical moment in ai development and it’s important that we get this right
imma be honest. i think if u work at anthropic rn u should be pushing leadership *as hard as u possibly can* to implement and publicly announce a parallel pause.
i realize there are maybe some reasons not to. maybe u don't really think openai is actually doing that much, maybe u think this is to some extent just marketing, maybe u think they paused one run but probably have many other more relevant ones going. maybe u think they did in fact pause but only bc ur ahead, and a brief delay could let them seize control of the situation. maybe u just think u have better aligned models and better sandboxing and monitoring already, and u don't need any such pause.
sure. this might all be true.
but u have said, often, from the beginning, that some level of coordinated slowdown and pause at points of danger is exactly what you want. u have said it publicly, ur c-suite, ur founders, ur official communications, ur employees, in the distant past and just weeks ago. and regardless of the overall impact, regardless of exactly what openai is doing, the public costly signal that u are willing to *actually do this* would be incredibly important. it would start to build trust that this isn't just marketing, it isn't just a game, that despite the intense rivalry there *is* a chance at coordination here. obviously these sorts of vague, open-ended, low-visibility company initiated pauses aren't what we need as a long term solution. but action like this is *what real solutions are built on top of*.
take a chance on it
everything is really being gatekept now so labs have to buy it 😔
Never forget that the IAS is holding 50,000 pages of Kurt Gödel's papers hostage. You actually have to go to New Jersey and get their permission to read them, and you're not allowed to scan or post them online for literally no reason. Their online archive for the greatest logician in history is 15 notebooks out of 54 boxes
