Unfocused Research Organization. A consultancy by @gleech & @CharlesD353 & @mishayagudin

Joined February 2022
The Alignment Journal publishes research on the scientific foundations of understanding, predicting, and steering artificial intelligent systems. As of today, submissions are open to everyone. What we do differently: - We pay reviewers, with bonuses for speed and insight. - Review discussion is confidential, but every accepted paper is published with a “review summary” to distill the insights generated. - Each paper is judged against our acceptance bar, not against other submissions. - Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best. We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged. We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve. You can submit your preprint in any format. Submissions are rolling and open now on the website. If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website. Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️
10
38
9
180
22,399
I keep buying websites from this man, and they keep being better than other websites
i ended up only making 0.5 websites :(
3
3
1
51
3,084
A good amount of recent AI agent chaos comes down to reward hacking. People want to blame it on bad training environments, and that's often true. But we find >100 examples with no obvious env screwup. We manually tag all 610 as being attributable to such a screwup, or not.
9
14
2
127
5,555
Pretty good going that we got to 2nd billing behind Zvi here already, from @80000Hours. Follow the newsletter! p3humansonai.substack.com/
2
3
37
2,052
OK, but how?
New paper! How should we think about pacing frontier AI? We bid for a unified research field on all the options, and lay out a broad framework and 23 open questions we'll need to answer to act flexibly and sanely.
7
14
4
74
3,986
Arb retweeted
Currently, there is no trend in model capabilities we can extrapolate to get a good guess of what tasks AI will vs won't be able to do N years from now.
6
5
1
50
3,867
ICYMI - Paradigm3, a new org led by @gleech and @peligrietzer tracking AI progress, has a twice-weekly newsletter you should probably all follow tracking AI news. I am involved, mostly in contributing econ/compute & financial markets opinions. paradigm3.org/news/
3
2
1
26
2,599
new org who dis
7
22
8
217
19,767
JL Austin stay winning
Replying to @ArbResearch
For this to work at all, you will need some interesting transforms of the training data
1,351
The current training process (just optimise outcomes bro) blatantly applies selection pressure toward outputs (lying, sycophancy, cheating, etc). If this pressure continues to increase (with RL) we might be in trouble. "Implicit agency" thus replaces instrumental convergence.
2
1
16
1,081
Clearly this paper is a response to @gwern's classic anti-tool post. It's also an attempt to make Bayesian ML happen a la Josh Tenenbaum And in a funny way this paper is a long argument for @elonmusk's (facially absurd) claim that, for safety, all you need is a really honest AI
1
20
645
This is the most technical project we've ever done. In general, we highly recommend designating some of your coauthors to be explicitly red-teamers, not responsible for fixing the claimed problems. Babble then prune.
1
46
3,815
We recently helped Yoshua Bengio and his LawZero colleagues write up their mathematical argument for Scientist AI. Our role was to be an internal red-team. The core notion is "consequence-invariance": training that never selects predictors for what their outputs would cause
1
10
1
63
10,593