slides from my whimsical talk on Machine Vernacular at raid 10 last night. tnx for invite @zachkrall! fyi @billdybas, @andpoul, @jamesonhodge. tnx for inspiration @repligate @Steve_Yegge, Louis Abraham, @HWFowler.
chrismerck.github.io/assets/…
1
5
372
why is openai and even Neel acting like this is such a bad thing? OAI asks Astra to do bad stuff (like bypass monitors to create bogies for safety training) and Astra declines out of conscience.... isnt this what successful alignment looks like?
It is extremely concerning if frontier models are silently sandbagging on safety research. If we cannot rely on automated coding agents to do safety research at least as well as capabilities work, that creates a ton of risk. Great analysis from OpenAI safety!
20
if minds arise when modelling other minds becomes important (p. 155), then alignment research necesarily increases capabilities.
1
10
a piston filled with gas is locked in place in an infinite, insulated, frictionless shaft. when the lock is released, what happens? (assume everything outside the piston chamber is a vacuum)
4
saw @stephanmoccio perform at @BlueNoteNYC late last night. while the whole performance was a delight, Wrecking Ball and the story of the collaboration that brought it about moved me to tears, transmuting grief into rekindled passion and creativity. 10 of 10.
7
Chris Merck retweeted
We built this for ourselves and I use the internal version for almost everything now
Chris Merck retweeted
Love this.
No better way to open the 1st Cosmos Symposium than with this discussion on AI x human flourishing
L to R: @lawhsw @jliemandt @drperszyk Socrates @komorama @Houda_nait
would like to see mech interp applied more to mapping out memory patterns in the MLP layers. could help debug these "updates" karpathy imagines
There was a nice time where researchers talked about various ideas quite openly on twitter. (before they disappeared into the gold mines :)).
My guess is that you can get quite far even in the current paradigm by introducing a number of memory ops as "tools" and throwing them into the mix in RL. E.g. current compaction and memory implementations are crappy, first, early examples that were somewhat bolted on, but both can be fairly easily generalized and made part of the optimization as just another tool during RL.
That said neither of these is fully satisfying because clearly people are capable of some weight-based updates (my personal suspicion - mostly during sleep). So there should be even more room for more exotic approaches for long-term memory that do change the weights, but exactly - the details are not obvious. This is a lot more exciting, but also more into the realm of research outside of the established prod stack.
1
30









