roominator (@roominator) header
roominator (@roominator) profile picture

implemented on a primate

Joined April 2020
Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)Photo by roominator (@roominator)
i think you lose something beyond system design when you don't code, not sure what it is
Replying to @HerrBultitude
Something that gets missed a lot is that the act of coding helps reinforce mental models for how systems work. I just end up spending a lot more time during white boarding now because I lose the intermediate step of coding.
2
3
110
another day of reading ai output
20
idk man i feel like its still just vaporware sometimes. AI development is sped up by TWO TIMES look at these BENCHMARKS the line is GOING UP even faster than my BLOGPOST said it would.
This report on AI R&D automation (RSI) is a big update towards ASI within 1-2 years. It says AI progress is sped up by RSI right now by around 2x. In AI 2027, this happens around March 2027. I was seriously wrong about RSI speed up; I previously expected much less in 2026 (around 1.15x-1.3x). It seems as though there was specific privately available evidence that would have changed my mind. I haven't seen an unredacted version of this report; this is purely about the public report. Because so much of the underlying evidence was redacted from the public report, I think conclusions about things other than the observed 2x faster rate of progress on benchmarks depend significantly on trust for the authors of the report. (E.g., conclusions about whether this speed up is from RSI depend on trust.) I do trust the authors of the report a lot to be careful and rigorous and generally do a good job. (Note that I also work at METR.) Less extensive redactions would greatly reduce this need for trust. Since in my current role I may have access to other confidential information that could be relevant to timelines one way or another, I unfortunately cannot state my all-things-considered views on ASI timelines or the probability of a software intelligence explosion right now. Here, I’ve done some analysis of the implications using my previous public statements. A 2x acceleration in overall AI progress from RSI corresponds to a much larger than 2x RSI speed up for AI software R&D (algorithms and data); I did an estimate using my public preexisting modeling of drivers of AI progress and the public report which yielded a very rough estimate of 4.5x for how much RSI is currently accelerating AI software progress (relative to no AI automation). In the AI 2027 scenario, this much acceleration happens around March 2027. Here are the worked details of this estimate based on my public modeling. There are two main reasons RSI speed up of AI software R&D must be larger: - Not all of AI progress is from better software (some is from scaling up compute in training runs). I've previously speculated that around 2/3 of AI progress is from software improvements which implies that 2x overall AI progress acceleration requires 3x faster software progress. - Diminishing returns: A fixed amount of progress requires more work over time, so to ongoingly maintain 3x faster software progress via AI automation, you need increasingly large speed boosts from AI automation (as in, speed ups relative to how fast software R&D would go without AI automation). Based on this report, it seems that there is a substantial amount of additional AI progress from automation that has occurred. We can estimate the effects of diminishing returns by first determining how many additional years of software progress we've seen (relative to the preexisting trend) and then estimating how much software R&D would slow down after a year of software progress if you held inputs fixed due to diminishing returns. I've previously estimated software R&D would slow down by around 2x for each year of software progress if you held inputs fixed (lesswrong.com/posts/FG54euEA…) and based on the report, it seems as though we've perhaps seen roughly 0.35-1.0 years of additional software progress incorporated into trained models so far (geometric mean of ~0.6). This combines to 3 * 2^(~0.6) = ~4.5. (Relevant public modeling and numbers I'm using: lesswrong.com/posts/FG54euEA…, lesswrong.com/posts/Jga7PHMz…)
1
61
wittgenstein remains undefeated teapawt.net/imjustnewatai/status/2…
Rumors I’ve been hearing. I’ll start with OpenAI, then Anthropic. OpenAI: The internal model’s reasoning jump comes from more aggressive RL, including training in games and simulated environments. Strategies learned there transfer to research mathematics and other fields. Navier–Stokes and ETH show the scale of the improvement. The clue is in the Synthetic RL hiring: self-play, simulators, and synthetic feedback. Anthropic: Same direction with its internal Claude model. It’s trained on long, difficult tasks in demanding environments, and the reasoning skills transfer to other fields. Its ETH result shows how far the improvement has gone. The earlier Universes listing and the RL hiring around mathematical reasoning align with that. Musk’s Digital Optimus hiring explicitly calls for measuring transfer from games into general computer use. That’s why his game-playing AI push matters: games develop abilities that work far beyond the games themselves. The interesting part is where those abilities show up. Each lab’s internal model applies reasoning developed through interactive training to mathematical problems that had resisted decades of work.
1
1
24
eveything is a game, we just had to build the gamer first
1
6
i think this is the cope we all tell ourselves to avoid logging off. ask yourself: should I care about the mood of the chronically online? teapawt.net/signulll/status/211330…
Replying to @AdamSinger
they’re really good for discovering the current societal mood.
20
people talk past each other nick: turns out the building blocks of this system are not inherently incorrigible in this way everyone thought yud: you still really haven't made a theory about how these systems work though teapawt.net/allTheYud/status/21108…
Me: Mechanistic interpretability sounds like a good idea, yes. If we could learn something about how an AI was thinking, in enough detail to predict in advance what it would think even as the thought departed from human, that would be a bare beginning on the sort of understanding you'd needed to get started on AI alignment that wasn't just about hammering on an AI from the outside with SFT and RL. AI guys: We found a neuron that lights up when the AI thinks of the Eiffel Tower! Me: Cool. That does sound like it'd be a prerequisite to getting started on the important part eventually, someday. AI guys: Oh NO! There are MULTIPLE neurons that light up when the AI thinks about the Eiffel Tower! And some of those neurons also participate in OTHER concepts! We're calling this unexpected development 'polysemanticity' and 'superposition'. Me: That sure sounds like a thing that would happen, yes. I cannot say that I am impressed by you acting surprised about it. Let me know when you've gotten far enough in mechanistic interpretability to get started on the important part. AI guys: We have found that with BIG AIs you can often point to a SINGLE neuron that lights up when the AI thinks of the Eiffel Tower! Understanding AI thought processes is gonna be EASY PEASY! Me: Cool. Let me know when you've advanced to the point of getting started.
45
I'm pretty sure we will run out of math problems. The limiting factor for creating them isn't postulating good problems, it's consensus on what is good.
22
roominator retweeted
TOTAL AND COMPLETE DEFINITELY ULTRA MEGA ABSOLUTE GARY ! MARCUS ! VICTORY!!!!!!!!!!!!!!!!!!! it all went exactly as predicted by Marcus (2019)
since many people are asking me about the new math results from OpenAI, here is my take. 1. It uses symbolic AI (Lean, etc) in addition to LLMs which is what i said for years we would need to do; this this confirms what I actually said (as opposed to various fictitious misrepresentations that run rampant around here ) 2. It still doesn’t mean they have AGI; the math stuff AFAIK does not generalize broadly, because you can’t use the same symbolic verification (and symbolic data augmentation techniques) in the open-ended real world. So yes I do feel vindicated wrt to neurosymbolic AI but no the G in AGI (general) is not yet solved. What I said in 2019 remains true: neurosymbolic AI is necessary but not sufficient; world models are also key.
1
2
25
4,629
roominator retweeted
Fucking incredible
Through the VOICE trial, Terry is using his Neuralink implant to help fine-tune a brain-to-voice interface for himself and others who can’t speak. He trained the algorithm first by miming speech as best he could, then by simply thinking the words and hearing them come out in his own natural voice. Powered by Grok Voice from @spacexai
22
118
8
2,896
395,808
people underindex how connected everything is: - no community if nomads can't get make a living and go extinct - twitter payouts stop if people can't pay for advertised products - assets stagnate or worse go if people can't buy services/goods
Oh for sure I think my only moat left post-AGI is: - Nomads.com community + data (now free membership) - Me being on @X with an audience - My levels.vc investments The rest will be completely and utterly wiped out with no doubt
1
38
to be clear I don't buy the premise. I think there will still be businesses of convenience that folks can build and I predict @levelsio will make most of his money from some future business (outside of his capital investments) rather than twitter
13
on one hand if they automate swe fully then there's an easy line to draw to losing your job. on the other hand, I don't think executive teams can understand or use a 'perfect engineer as a service', and will continue to pay people to understand things teapawt.net/BumrahBachi/status/209…
I think the music has stopped The quoted tweet is from february this year but progress has been such that at this point I think your career is meaningless if you don't own substantial capital, do something physical or work at OpenAI or Anthropic. teapawt.net/BumrahBachi/status/201…
1
39
at some point the bottleneck becomes human ability, e.g. even with the perfect engineer as a service, you need >1 human engineer to sufficiently cover an understanding of all projects
8
i want to do this so badly (i own an apple trackpad) but a mouse is so much better unfortunately teapawt.net/levelsio/status/209397…
Replying to @tdinh_me
I went full MacBook track pad
33
depends on what the output is, if you maintain a model (classical statistics or otherwise) that is shipped to prod and does something real to increase revenue it's not fake imo
Data Science is more fake than any email job
83
yeah it's thinking machines
Why the fuck is this stealth model so good at frontend?
4
109