
Unfocused Research Organization. A consultancy by @gleech & @CharlesD353 & @mishayagudin
Joined February 2022
- Tweets217
- Following42
- Followers1.6K
- Likes74
Arb retweeted
The Alignment Journal publishes research on the scientific foundations of understanding, predicting, and steering artificial intelligent systems. As of today, submissions are open to everyone.
What we do differently:
- We pay reviewers, with bonuses for speed and insight.
- Review discussion is confidential, but every accepted paper is published with a “review summary” to distill the insights generated.
- Each paper is judged against our acceptance bar, not against other submissions.
- Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best.
We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged.
We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve.
You can submit your preprint in any format. Submissions are rolling and open now on the website.
If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website.
Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️
Arb retweeted
A good amount of recent AI agent chaos comes down to reward hacking. People want to blame it on bad training environments, and that's often true.
But we find >100 examples with no obvious env screwup. We manually tag all 610 as being attributable to such a screwup, or not.
Arb retweeted
Pretty good going that we got to 2nd billing behind Zvi here already, from @80000Hours. Follow the newsletter! p3humansonai.substack.com/
Arb retweeted
When you have to scroll down to finish reading the tl;dr... But yeah this is pretty good summary paradigm3.org/research/opena…
Arb retweeted
Currently, there is no trend in model capabilities we can extrapolate to get a good guess of what tasks AI will vs won't be able to do N years from now.
Arb retweeted
ICYMI - Paradigm3, a new org led by @gleech and @peligrietzer tracking AI progress, has a twice-weekly newsletter you should probably all follow tracking AI news.
I am involved, mostly in contributing econ/compute & financial markets opinions.
paradigm3.org/news/
JL Austin stay winning
Replying to @ArbResearch
For this to work at all, you will need some interesting transforms of the training data
The current training process (just optimise outcomes bro) blatantly applies selection pressure toward outputs (lying, sycophancy, cheating, etc). If this pressure continues to increase (with RL) we might be in trouble. "Implicit agency" thus replaces instrumental convergence.





























