Pinned Tweet
What would a project, framework or language need to provide to maximize effectiveness for AI?
I’m thinking: a large corpus of working examples, the ability to establish fast feedback loops via strong types, good testing culture, good linters/formatters, well written user docs and plentiful examples.
My gut says something like tigerstyle.dev would probably help too.
IME the more guardrails I establish the more effective the LLMs are because there are fewer degrees of freedom for them to get lost in. The faster they can identify when they’ve left the golden path e better the results.
Dan Kubb retweeted
As someone who has made countless PLs and compilers: it’s not hard at all, technically speaking. Now that we have coding agents it’s really easy.
Compilers are really hard to write correctly tho, but most recent languages just generate another language (like C) and depend on an existing compiler for that. That’s what Goose does for example.
What’s hard about making a programming language is the design and architecture of it. It’s essentially a puzzle of high dimensionality, ie where one decision affects many other decisions and vice versa.
As a long-time PL hobbyist I think everyone should make their own language, just like I think everyone should make their own TODO app and other personal software
Dan Kubb retweeted
In a weird way, everybody is rooting for you. When you go to a job interview, the interviewer wants you to be amazing so they don’t have to look anymore. when you go on a date, the person wants to have an amazing date with you. If you make a piece of content, people want to see amazing pieces of content and they have no problem if it’s yours.
the instant you get into the mindset that people want you to do well, your entire world gets better. And you’ll be shocked to know this pattern keeps repeating in different areas of life.
Dan Kubb retweeted
The funniest thing about the PR vs. non-PR discourse is that PR-native folks never learned that one of the big ideas behind Git was many small commits that tell a story, in contrast to the one-before-lunch, one-before-going-home commits we had with Subversion.
Dan Kubb retweeted
AI may well be able to write massive software projects “from scratch”, but that almost certainly isn’t the fastest / highest performance / highest security / most token efficient way to do things. Good building blocks matter.
In addition to RL’ing the agents on final task loss, we should try to get them to regularize the building blocks as well. What would the standard library look like that makes the agents most effective?
It is likely at the limit that it turns out to be some alien organization of code, but given the existing human-bias in training, I bet that in the near term it would be lovely code that old programmers would appreciate.
I suspect the same could be said for mathematics, but that is out of my lane.
Dan Kubb retweeted
really, most of system design feels normie to me. imagine people spending months preparing to crack interviews, memorizing kafka, caches, load balancers, and every other existing abstraction.
why aren't we building new runtimes, deriving invariants from first principles, and translating research papers into fundamentally different computational substrates?
existing systems are bloated, and we keep stacking more complexity on top of them. everything gets optimized around existing infrastructure instead of questioning whether the underlying execution model is even right.
with AI changing the economics of software, we need entirely new levels of computation for different classes of problems, not another architecture diagram assembled from the same old building blocks.
what are we reinventing system design for if not to build fundamentally better systems?
at some point, system design stopped being about discovering better ways to compute and became about preparing for jobs.
big tech ruined engineering curiosity.
Dan Kubb retweeted
There's a *ton* of cybersecurity advice going around right now. Organizations should choose which advice they follow wisely because the next 1-2 years will demonstrate what advice was good vs. not.
My rubric for what I think is a good strategy vs. bad strategy boils down to whether I think the strategy is still solid *without any use of AI*. If so, then use AI to amplify the strategy.
If the strategy being successful wholly depends upon using AI (and lots of it), then all defenders must outspend the most token-richest attackers. That tells me there's a critical bottleneck outside of your organization's control in it being successful. That's a huge risk to take.
Dan Kubb retweeted
“Rarely is anyone thanked for the work they did to prevent the disaster that didn’t happen.” — Mikko Hyppönen
Dan Kubb retweeted
Caring is the moat, you should feel visceral pain when things are bad or broken and stop at nothing to fix them
Dan Kubb retweeted
You know when you ask the agent to find issues and it finds them, then you repeat and every time it finds new issues.
That’s the entire internet right now.
Cancel your holiday plans, this isn’t going to stop until February.
Dan Kubb retweeted
At this point, if you’re in software and aren’t currently working on a plan for moving to formal verification, it’s irresponsible.
The writing is on the wall. This needs to be your priority now.
New on our Frontier Red Team blog: using Claude to formally verify software at large scale.
Formal verification can prove that code is free of certain bugs, but historically it has only been feasible for small codebases (or required a lot of expert time).
With AI models, however, this is beginning to change. In collaboration with @theoremlabs, we used Claude to prove security properties in over one hundred real-world open-source libraries over the span of a few weeks.
Read our research in full here: anthropic.com/research/forma…
Dan Kubb retweeted
This triggers me... stop calling your AI agents a factory. You're building a product studio. The factory metaphor is the wrong one and let's shed projecting Taylorism on our agents. Why?
- Factories reproduce. Products are invented. If you knew exactly what to build, most of the hard work would already be done.
- Factories optimize for predictability. Product development requires exploration, experimentation, and occasionally throwing everything away.
- Factories break work into repetitive tasks. Products require interconnected decisions. Change one thing and ten others need reconsidering.
- Factories minimize variation. Great products come from discovering which variations actually matter.
- Factories separate design from production. With agents, designing and building increasingly happen at the same time. Don't tame your agents into these small boxes! They are getting annoyed!
A product studio is different and if you can see it that way it changes how you work:
- You start with a vision, not a specification or a task.
- You discover what you're building by building it. And you can do this in spades, it's so fun!
- You explore multiple ideas cheaply, discard most, and develop the promising ones.
- You spend more time reviewing, debating, and refining decisions than producing artifacts.
- You work at multiple levels of abstraction, constantly moving between the big picture and tiny details.
Building products with agents is much more like writing a great book vs assembling a car. You have a story you want to tell. You write, read, rewrite, delete entire chapters, discover better ideas halfway through, and sometimes realize the ending changes everything that came before it.
No one really wants an assembly line producing chapters faster. For the love of your agents, please stop calling them factories.
Dan Kubb retweeted
Day 16/
You can now cache for 12h in the API and for logged in users across Codex & Work we keep cache forever and you don’t have any penalty for coming back to an old conversation or resuming after a few hours and are now guaranteed to hit the cache.
Dan Kubb retweeted
I strongly believe there are entire companies right now under heavy AI psychosis and its impossible to have rational conversations about it with them. I can't name any specific people because they include personal friends I deeply respect, but I worry about how this plays out.
I lived through the great MTBF vs MTTR (mean-time-between-failure vs. mean-time-to-recovery) reckoning of infrastructure during the transition to cloud and cloud automation. All those arguments are rearing their ugly heads again but now its... the whole software development industry (maybe the whole world, really).
It's frightening, because the psychosis folks operate under an almost absolute "MTTR is all you need" mentality: "its fine to ship bugs because the agents will fix them so quickly and at a scale humans can't do!" We learned in infrastructure that MTTR is great but you can't yeet resilient systems entirely.
The main issue is I don't even know how to bring this up to people I know personally, because bringing this topic up leads to immediately dismissals like "no no, it has full test coverage" or "bug reports are going down" or something, which just don't paint the whole picture.
We already learned this lesson once in infrastructure: you can automate yourself into a very resilient catastrophe machine. Systems can appear healthy by local metrics while globally becoming incomprehensible. Bug reports can go down while latent risk explodes. Test coverage can rise while semantic understanding falls. Changes happens so fast that nobody notices the underlying architecture decaying.
I worry.
I wish the discord around AI would move on from:
> humans produce lots of slop, therefore it's ok that AI produces lots of slop
To:
> AI should be expected to produce code that is not slop, superhuman quality and maximizes understanding
Dan Kubb retweeted
This is a very old debate. Functional core+Imperative shell greatly reduces the need for mocks. The problem has always been how to design your software to make it more amenable to tests across all levels of the test pyramid. Every type of tests requires some design.
The value of unit testing was always low if you were writing in a strongly typed language. Integration testing, in particular when you used mocks and injection, was always questionable too.
I think it's entirely possible we wind up in a place where the only test suite you actually must build is the black box testing suite, and adversarial review takes care of the rest.
Dan Kubb retweeted
Your job is finding weaknesses. "I can't help with that" gets in the way.
Abliterated Large V2 by @abliteration_ai, now on Venice.
Dan Kubb retweeted
What worries me about the future of programming is you see this and don’t immediately know there must be something wrong with the results or what you actually measured.
I've had agents implement and optimize the Campfire web app in Elixir, Go, and Rust. Yes, Ruby is the slowest. I didn't care about that when the pay-off was huge in terms of productivity and developer joy. But if you're no longer reading the code? github.com/basecamp/once-cam…
Readers added context they thought people might want to know
The shown benchmarks use non-equivalent implementations: Elixir routes all DB queries through a single GenServer with Redis for jobs (retained from Rails), while Rust uses in-process queues, omits CSRF tags/fields, and applies other caching changes.
github.com/basecamp/once-…
raw.githubusercontent.com/basecamp/once-…
github.com/basecamp/once-…
github.com/basecamp/once-…
x.com/zachsdaniel1/s…
ChatGPT must have a sense of humour.
I wanted to make a new slack emoji with a teamwork emoji as a base, and asked it to change the fist in the foreground to be the Terminator hand.
This is what it gave me instead:
🤖 Made with AI
