San Francisco, CA
Joined January 2008
- Tweets11.6K
- Following512
- Followers5.5K
- Likes1.5K
There needs to be a benchmark that mixes common acronyms with a few undefined fictional ones to test whether the model hallucinates the acronym's expanded definition.
I see this even on frontier models, even when they have the tools available to search for and define the acronym properly.
Software dev is wildly fluid compared to other engineering disciplines. The ability to rapidly iterate alters the fundamental approach of the practice.
Dismissals of AI-maxxing oft include a judgment of the work being done as "not serious", but this misunderstands software.
Faster iteration means more nimble responses to market demands. Software teams can focus on features one week, performance the next, and reliability the week after that depending on criticality of each relative to product market fit and overall business maturity.
In Pluribus, an alien intelligence usurps overnight and dissolves all previous divisions, establishing a semi-utopic society with regretted casualties along the way and a small group who refuse to partake of the new order.
So it's about AI and the software industry, basically.
I feel like the more powerful agents get the more valuable software engineering experience and expertise becomes.
Building software is now about providing concise, properly scoped goals and verification criteria. The people best suited to do that? Expert software engineers.
I used to worry about the junior to senior pipeline in an AI world and I'm still not sure exactly how it will work but I'm more convinced now that it will work.
It'll be a different set of skills but the same hard-won wisdom can still be acquired by other means, I think.
I find myself frequently telling LLMs to explain things like I'm a "semi-technical exec". It's actually not a bad way to get more comprehensible technical writing out of them. 😂
We live in an age of wonders. If you are a software engineer who mostly knows what you're doing there's essentially nothing you can't create and figure out now.
The creative potential of a single human is rising on an unbounded exponential curve. It's overwhelming.
Michael Bleigh retweeted
I've published a new, general-purpose terminal specification that gives any program a way to tell the terminal what it's doing: idle, working, waiting, finished, or failed, and why. It is easy to implement on both sides. I've written more about why: mitchellh.com/writing/progra…
The problem this solves is generic and applies in compelling ways to all sorts of use cases (e.g. package managers, build tools), but it really rears a hideous head when it comes to AI and the "agentic inbox" problem.
I found over 250 different agent orchestrators (with various goals) that all individually implement a heuristic based approach to detecting whether things like Claude Code are working, blocked, done, etc.
If you look at @herdrdev's commit history, you can see around 10 compatibility changes to detect Claude Code within just the past 3 months. (This isn't a dig at Herdr! They're doing great work with what they have, and everyone else is doing the same thing).
Heuristics are not the way. And over-indexing on agentic use cases isn't the way, because this problem is really interesting for other tools too: Homebrew, Terraform, Cargo, etc. etc.
Proprietary protocols and out of band APIs are not the way. They have an O(N) integration problem and require extra work to work over SSH or within local VMs/containers.
The program status specification (OSC 7501) solves this in a reliable, deterministic way that is general-purpose and can be easily adopted by the entire industry.
Read more: mitchellh.com/writing/progra…
"This in-app agent could have been a skill" is the new "this meeting could have been an email". If people at your company want to build agent functionality that can only be used inside your app/product, tap the sign: xitter.cf/ASpittel/status/209088…
Unpopular opinion: plan mode is still good and correct. Now that models are more powerful and do longer horizon work it's more important to be aligned up front.
A dedicated plan artifact is easier to read and review than inline chat content.
I would estimate a high percentage of all software (80%+) would be totally fine with only automated reviews. Because 80%+ of software written is relatively low risk or changes are easily reversible.
AI can free us to focus on the changes that actually do need expert attention.
It used to be that incomplete software was simply missing many things that needed building.
Now incomplete software has all the pieces, but the pieces are bad (i.e. slop). Finishing a project is becomes more about removing bad ideas and polishing good ones.
This isn't good or bad, just different.
I'm seeing SO MUCH Jev in my feed. My theory is this: most people don't want to train models, they want to build things.
So just like how LLMs broke through to the mainstream because they didn't need to be trained to be useful, such is true for Jev and classification tasks.
