Unfocused Research Organization. A consultancy by @gleech & @CharlesD353 & @mishayagudin
Joined February 2022
- Tweets217
- Following42
- Followers1.6K
- Likes74
Arb retweeted
The Alignment Journal publishes research on the scientific foundations of understanding, predicting, and steering artificial intelligent systems. As of today, submissions are open to everyone.
What we do differently:
- We pay reviewers, with bonuses for speed and insight.
- Review discussion is confidential, but every accepted paper is published with a “review summary” to distill the insights generated.
- Each paper is judged against our acceptance bar, not against other submissions.
- Papers under review are listed publicly, and researchers can nominate themselves to review, so each paper can reach the reviewers who know the topic best.
We seek work that builds theoretical or otherwise principled analyses of agency, robustness, incentives, interpretability, generalization, and long-term behavior, and whose significance survives the deletion of any particular model, dataset or benchmark. Empirical investigations of such ideas are welcome, and negative results are encouraged.
We are especially interested in approaches that could apply to future superintelligent systems that interact and self-improve.
You can submit your preprint in any format. Submissions are rolling and open now on the website.
If you are instead interested in reviewing or serving as an action editor (basically: an area chair), create an account on our website.
Links to submit, our scope and the reasoning behind the Journal are in the reply ⬇️
Arb retweeted
A good amount of recent AI agent chaos comes down to reward hacking. People want to blame it on bad training environments, and that's often true.
But we find >100 examples with no obvious env screwup. We manually tag all 610 as being attributable to such a screwup, or not.
Arb retweeted
Pretty good going that we got to 2nd billing behind Zvi here already, from @80000Hours. Follow the newsletter! p3humansonai.substack.com/
Arb retweeted
When you have to scroll down to finish reading the tl;dr... But yeah this is pretty good summary paradigm3.org/research/opena…
Arb retweeted
ICYMI - Paradigm3, a new org led by @gleech and @peligrietzer tracking AI progress, has a twice-weekly newsletter you should probably all follow tracking AI news.
I am involved, mostly in contributing econ/compute & financial markets opinions.
paradigm3.org/news/
Arb retweeted
I'm helping out with a new funder! Short applications, deadline in two weeks, decisions hopefully in four weeks. $50k cap because we're covering stuff other people cba with
Arb retweeted
In the last 5 months, 18 Erdős Problems were solved, for the first time, by AI systems working alone. How should mathematics respond?
Prof Václav Rozhoň & I are building a journal to filter this flood of AI maths at scale. We have a plan, funding, & credits; now we need a board
Arb retweeted
seems you're right (Erdos problems, human vs teaming vs AI): +17% diff, nearly significant at 5%
thanks to @nc_znc
Arb retweeted
Join us as we publish hermits, informants, translator-traitors, forum freaks, discord archives, pirated plays, psychiatry bloggers, groupchats and television recaps
Grateful to join @suspendedreason, alongside @ctbeiser, to further Not Nothing's first decade of work towards new forms of reading.
Arb retweeted
Impressive project requiring an almost egregious amount of handwork
Arb retweeted
PSA (thx to @abstractapplic & Noosphere89¹):
LLMs often think they're in an eval when they aren't, probabilities reported: 15%, 15%, 2%, 3%, 25%–40%, 15%–25%, 60%–75%, 10–15%, 5-10%, 70-80%, 55-70%
This is way higher than what a baserate should say, which is ~10⁻⁴–10⁻⁷
Arb retweeted
Note that if this was true the labs would be wholly dependent on Nvidia and the hyperscalers. There's ~no world in which the labs own most of the compute themselves in the next decade.
In some takeoff scenarios, the value of compute becomes too high to sell. The opportunity cost of selling the chips just becomes too high. Eventually it becomes more profitable to keep chips rather than to sell them to anyone else. Not long after that, it may become vastly more profitable still. At that point, compute probably starts to look more like political currency.
This is one reason most frontier labs are working on their own chips. At a certain point, no one sells freely anymore. There is a threshold where even Jensen stops selling in the ordinary sense. Everyone already established probably gets their yearly allotment of spice, and that's it. And no one resells. Either your internal projects are too valuable, or you lease your capacity strategically to whoever is in first or second place in a takeoff kingmaker gambit. Reselling would probably also be death-penalty illegal by this point. And what is the point of long-term contracts anyway if your internal reports say human employment, and even money itself, may not function or look anything like the way it does now in as little as ten years? Maybe even five. Things start to break a little. People stop caring about some consequences if they don't see them carrying any weight in the new world. And as the new world starts to seem more real, the old rules start to matter less. What does a fine mean in a post-scarcity society? It means nothing.
This is why Elon builds TeraFab. And this is why Sam Altman wanted to build Project Tigris. After a certain point on this trajectory, the only reliable fab is your own fab. Compute becomes so valuable that rules and incentives begin to break. Alliances break. Contracts break. Rumors of nationalization probably start around then too. It is easy to talk about building a god in your lab as long as no one takes it seriously. The moment they do, the government becomes much more involved. If you do believe in a takeoff scenario this decade, then the USG will have a very large role in this story, and probably a lot of influence on any takeoff trajectory.
Arb retweeted
I am excited to say I'm joining @gleech at @ArbResearch!
Arb and Gavin have done some very cool projects over the years, and I am very keen to get stuck in and contribute.
If you are interested in working with us, email charles at arbresearch.com
Arb retweeted
Incredible piece. Part review of @dwarkesh_sp & @gleech 's The Scaling Era, part tour of the fascinating intellectual and cultural milieu of AGI-pilled Bay Area 20-somethings.
americanaffairsjournal.org/2…
