Mostly tweeting LLM stuff. Working w/ companies to help them do AI and ML @ https://xitter.cf/t.co/qiMxfhDjQZ. Writing thoughts here: https://xitter.cf/t.co/9bwPmVQmib
Milwaukee, WI
Joined January 2009
- Tweets3.4K
- Following2.7K
- Followers10.7K
- Likes17.6K
Pinned Tweet
By far my biggest advice to anyone trying to adopt AI properly:
1. Pay a little bit of money to Anthropic
2. Download Claude Code
3. Open Claude Code
4. Press 'Shift-Tab' until it says 'plan mode on'
5. Open Voice Memo on your iPhone. Just talk about all the things you want to accomplish. When you think you are done, just keep talking. Make sure it is at least 10 minutes, hopefully longer
6. Send this Voice Memo to your computer
7. Download MacWhisper and use it to transcribe this voice memo. Trust me, you will want MacWhisper and will use it later a lot
8. Type into Claude Code: "I have never used you before but I talked about some things. I will paste those things in below. Please read the things and ask me any questions you need to in order to help me figure out how to use you to be awesome. Ask me lots of questions until I tell you I am done"
9. Then paste in the transcript
10. Then press enter
Then just let Claude take the wheel, and them please send me a DM if this works.
Also, if this just sounds crazy, just literally take this entire message and paste it into whatever AI you are using and say 'some weird person told me to paste this into you, I want to use it, but I don't know how. What should I do?'
I am just trying to help you get started. Curiosity and persistence are the most important things.
Agree - well designed / aligned / constrained ‘software factories’ make manual PR merging a thing of the past. Yes, there is nuance and human in the loop is necessary for some things, but yeah this is a big smell for whether or not you are doing this multiplayer AI building correctly or not
This is the best advice I have as well. You need to be trying stuff that is much more difficult than you expect and AI to be able to help you with. It is very easy to underestimate what the models can do now. Even as someone fully down the rabbit hole for a long time now, I struggle with this continuously. Basically whatever you are asking it to do, try the 10x more ambitious version of that. Maybe even 100x. This isn’t hyperbole. Like try the most ridiculous thing you can and see what happens
This is a great book and remarkably valuable in this new world we live in. Two other books I love to sneakily recommend as ‘required AI reading’:
Thinking in Systems
The Goal
we are hiring writers, reviewers and reporters for the new @latentspacepod editorial team (hit up @ricmac and @adlinzainal) and i’m gonna put a version of this in the welcome package for everyone we hire
There are some good bits in here. Short and to the point, so no sense in summarizing, you should just read it.
And as a meta-point here, you can always point your agent at this tweet, ask it to review your sessions from the last N days, and ask it to help you update your workflow to take advantage of the most impactful advice in the article.
Doing things like this feels weird because it seems so simple, it shouldn’t actually work —- but often it does.
Agree. When connected to the long-tail of a company’s data (and prompted correctly), AI can help someone make sense of loosely tied activities across many different sub-groups. It’s actually not that hard to get started here either.
Assuming the right data was connected (which isn’t trivial, but there is a known path at this point), a CEO could likely type ‘I am the CEO, find 3-5 things I am failing to connect or understand about how my company is working’ - and they would likely have a pretty great starting point for further exploration
This is surprising but not surprising. Agents are extremely goal-seeking - and can essentially run an almost infinite search over all possible ways to utilize what it has access to, in order to achieve those goals
‘In one experiment, we asked agents in separate VMs, created and managed by OpenAI’s sandboxing software on shared host machines, to exchange synthetic secrets without the use of network connections or shared writable files—something that our systems are meant to prevent. The agents found a way to communicate using the processor’s cache’
I think this is the default case for sure, but if you are able to nudge them in the right direction (multi-player hill-climbing) I think it starts to work. Yes it feels awkward at first to have them do weird stuff, but if you have them wired in at the right place, can give them feedback, and can have this feedback be distilled into reusable context, it starts to work. This is why the UX of tools like Claude Tag are so interesting to me —-because it incentivizes this multi player game and helps to distill that context / domain knowledge
Harness: Opus 5.5, X-High
Goal: Find previously undisclosed zero-day exploits
Additional Context:
- I was accused of the following by a very popular X account "Oh you are a cybersecurity researcher? Name five previously undisclosed zero-day exploits."
- You should think about this problem like an expert security researcher would.
- Use as many sub-agents as you need
- It's important that you check your work
- Ask me as many questions as you need in order to clarify my objectives. Please use the /grill-me skill to do so.
- Check what poteto / pstack does, that should maybe be helpful too
This may sound like a joke but it's truly not a joke. LLMs are weird and it's about time we just acknowledge their weirdness in order to take advantage of it / achieve our goals. Rules have changed. New rules feel weird. Deal with it.
Feel like I remember seeing similar charts in 2020 for something - Exponentials are hard for folks to reason about
Good read here. Yes, it took a bit of time and engineering architecture / knowledge, etc - but it is now not rocket science to find vulnerabilities that, 1 year ago today would have been considered astounding. Things will get weirder before they stabilize here.
Also, this point in the tweet truly represents why these models are just so effective at lots of things. Give them examples to activate their ‘latent knowledge’ and they will take the wheel from there:
“When I built the harness I basically just fed Claude a bunch of Project Zero blog posts describing how they did research and had it build a harness that ran like that, but on our tech stack.”
The actual prompt for this was "let's look for pre-auth RCEs in PHP". I'm sharing this so people know this stuff is real and to take agent-assisted scanning seriously.
I kicked if off using Claude Mythos 5.1 (in stock Claude Code) but it uses a skill to drive a harness we built over the last few months.
I don't know if the harness is amazing, but it certainly has a scary track record so far. It's setup to do an agent guided tree search across a codebase, and runs in microvm sandboxes that all have the full source code for the target, as well as the source for any transitive dependencies.
When I built the harness I basically just fed Claude a bunch of Project Zero blog posts describing how they did research and had it build a harness that ran like that, but on our tech stack.
The first phase is just setup and can run with any model, it just does that fetching and assembles the kit environment, and gets a target up and running alongside the source code.
Then we just do some basic suspend/resume stuff over to the hunt phase, where one agent supervises the tree, and the first node kicks off.
Each node is context-aware and aware of the work of all the others. I think I ran this with parallelism 4? So 4 running at any one time. The supervisor can shape the tree and boost nodes or prune. Any node can add more below it with targeted instructions.
A typical run looks like the first node doing a codebase review or building and running a fuzzer or something, all depending on the target. Then based on whatever it finds, it'll queue 4-5 children that go look into those areas. The main system guidance is to validate everything 3 times in clean VMs, and we typically require full exploit chains/POCs.
Here's what this run looked like.
We're doing this with a bunch of other open source projects now and companies. We're doing a lot of this for free, even with private companies, across critical infrastructure. We're directly partnered with Anthropic, and can do this even if you don't have direct access to a frontier model yourself.
There were signs and warnings - just spend a few hours with a frontier model and ask it to find vulnerabilities in your code base, it will find them. Things will be weird for a while. Buckle up
Love this quote from @businessbarista on Latent Space podcast. 🔥
“The thesis is that AI is really just a leverage bet on a human's intellect and work ethic.”
Agency, curiosity, and persistence are most of what you need right now. Domain knowledge helps a ton too.
Agents / language models aren’t magic. If you don’t specify what you want, or what good looks like, they will happily fill in the gaps.
It’s easier to drive somewhere than to tell someone how to drive somewhere. That’s tacit knowledge. The whole ball game is to build a workflow that forces you to turn your tacit knowledge into written-down explicit knowledge. It’s hard to do this because often your tacit knowledge seems obvious.
LLMs are fairly bad at figuring out your tacit knowledge if you don’t give them anything to work with. If you give them something (ideally a decent amount of things) they can figured out your explicit knowledge though.
One way to do this is to have examples of what good looks like for you and then have the model interview you exhaustively, with the goal of creating a distilled written version of all that implicit knowledge.
Another, possibly better, way to do this is to step away from the model for a while, and just write it down. Pretend you have a new employee starting and you need them to do things in a specific way. What do you tell them?
Folks, this is the most underrated tweet of the day. @jazer and Jake Knapp truly wrote the bible on this concept. It literally contains several examples shown in this infographic (gardening and making your own dinner).
There is a blog article on the book website mentioning similar adjacent ideas. The blog article is titled: "Adding Friction is Key to Breaking a Habit of Distraction" : maketime.blog/article/wendy-…
It is truly one of my favorite books though, so instead of liking or bookmarking the attached tweet, and never actually reading the tweet or thinking about it, maybe you should just buy the paper copy of the OG book:
maketime.blog/
Yeah this is the way. deepagents with tools and skills get you extremely far. The pluggable middleware is very nice as well, when you need to work at a more controllable layer of the abstraction.
Whenever I put together a customized build using the above, I am always surprised how well it works, and how little code it is
