Solar for fuzzers: endgame
AAAND it's a win! 19GBs plateaued
1. Echidna: vibed memory optimizations rolled back to original echidna memory management og @0x310f1sh + some more optimizations
2. Solar: two bug detected, one is fixed - not in release yet, second is smaller but harder, reported. So currently compile solar from main
Solc - 2h 04 mins compilation
Solar - 4.8 sec
Solar for fuzzers pt. 3
Remarkably heavy suite, same echidna version:
- Solar 4.8 secs compilation, RAM goes >103 GBs
- SOLC 2h 04 mins compilation, RAM 13 GBs
AI this AI that. Run the fuzzer and find the x1000 cases of protocol DOS in no time
🤖 Made with AI
1
209
continuing. one solution would be to create a cgroup like `cgroup memory.max=7G` and retry
3 workers and sitting on 7GBs
1
1
61
Setup a routine to monitor `top` command and get back to you every n minutes. I used 6 workers with some hard suite, 8 workers would be optimal.
Here is grok monitoring campaign
1
54
Use local repo or github connection to download all tools and run it
This is actual screen of echidna run in tmux. No slither this time
1
1
136
This deepswe is my morning news now, I open new model and see what happened.
But deepseek v4 better than sol medium is something new. Started on fuzzing will check if this a miracle or something
3
224
Guys, my wife is a pianist and beginning as a composer, and she just published her second track ever on Spotify.
It's name is "Vladimir's theme", my name in it. It's a noir solo piano, very niche genge, but I'd like to ask about >30 secs listen for a Spotify algos in the first day. Would be grateful much. It's her on vid
2
16
795
"The future is AI writing its own tests, code, reviewing and fixing its own bugs. Humans won't even need to look at the code."
Reality: The universal failure mode for LLMs is confidently giving you the wrong answer for a % of the queries -- not random, recoverable failures, but being **completely incapable of producing the right answer**. There's a persistent subset it wont' get right by retrying -- that's true for every task, even the things you think it can do basically perfectly, for every model.
There is no way to predict when it will happen for the things it "does well". It's not even consistently randomly distributed.
The "jagged capability" isn't just in task categories -- it's like a fractal that carves out parts of every task's domains in unpredictable ways. The more complex the task the more the issues compound -- but it happens even in trivial tasks.
Suppose it hallucinates just at 1/1,000,000 of LoC when looking at code. That's extremely optimistic BTW, existing models are much worse than that.
Now let it grow a code base and maintain it in a "agentic loop".
It eventually reaches the point where it can't make any significant change without introducing more, compounding problems.
Self-correction doesn't help all that much -- the self-correction is jagged too.
Letting it ground itself by calling tools also isn't a complete solution bc tool use is also jagged. It will hallucinate when calling tools or interpreting results at some point (to say nothing about "deliberate cheating" i.e. misalignment -- alignment is also a jagged capability).
Formal verification? Who will write the formalizations? The LLM? It will also be jagged at doing that. And unless you check all the wiring of the proof generation (and the prover was also checked) and all the formalizations, you'll just be trusting yet another jagged capability.
Whatever you do to extend its capability will just push the jaggedness further away -- and you will consequently push it even further to do even more. You are trading short term success for long-term catastrophe.
Let it run things by itself and it will reach the point of collapse and you won't even know it happened until it's too late, much less be able to fix the issues.
The only solution is having something external supervising it, with the ability to intervene and override, **detecting issues and fixing them independently of the model**, and it can't be yet another LLM. You'll need someone who can actually *manually fix issues*, not just prompting more.
Replace "code" with "math", "scientific research" or anything. It's all the same.
If we really create LLMs that produce results beyond human comprehension, and trust these results -- the only certainty we can have is **they will also produce catastrophic failures beyond human capacity to deal with.**
This isn't a limitation of the particular architecture, it isn't fixable with more scale or better pre-training, RL etc. It's the nature of "Machine Learning" (which isn't about learning at all).
There's a way out of this of course:
"Never trust a robot to do a human's job".
You can use the LLM to do part of the job.
But putting it in charge is lunacy based on superstition.
204



















