Jew Peter retweeted
i hypothesized that a new LLM pipeline
would make a better trading agent.
THE RESULTS ARE IN:
1. dramatically correct in some cases
2. dramatically incorrect in other cases
let's discuss
\
\
\
The core change was moving from a single LLM call to generate a trading hypothesis to a multi-step LLM approach.
The assumed benefit is that we could apply more intelligence and parameterization to the hypothesis generation pipeline that @ASYM41b07 uses to ideate trading strategies in a way that would produce more profitable trading strategies.
This worked.
It also failed.
The multi-step process introduced new features/approaches to support the LLM calls:
🤠 alpha rotation (to force ideation diversity)
🧐 feature agnostic ideation w/ remapping
🤓 market-informed signal logic
🤪 market-informed risk parameters
🥱 strategy-aware rate limiting
Much of this was done in response to analysis of strategies produced from our first Benchmark (B1) where we requested ASYM to generate, backtest & eval 140 strategies.
After we made the changes to the LLM hypothesis-generating pipeline, we ran an additional Benchmark (B2) to compare the efficacy of our changes.
The good (as hypothesized):
- Avg PNL improved bigly: -3.2% -> -.62%
- Avg Win Rate UP bigly: 22.7% to 32.8%
- Drawdown down bigly: avg max DD 2.97% -> .55%
- Diversity Improved smoly: avg jaccard .082 -> .067
The bad (unexpected & undesired!!!):
- Strat prod funnel shrank: 70% success rate to 49%
- absolute PnL right tail clipped: 3.3 -> 0
- Sharpe mean fell: -.46 -> -.73
key takeaways:
a) we can definitively influence the trading capacities of the system, and on our first go, we've shown we can make strats move up and to the right. this is a big demonstration.
b) atomizing the hypothesis generation pipeline made the system less effective at producing strategies that are backtest-able/functional; in isolation, this isn’t a massive issue, mostly adds cost overhead, and should be reasonable to improve.
a) a real issue that we'll have to focus on: our changes capped any strategy’s capacity to produce profits. the new risk setting logic (re TP/SL/hold-time) made Strats safer but also capped winners. Win rate went up but profits never ran—— Benchmark 1 actually produced more profitable Strats than Benchmark 2!! Given we can produce n strategies on demand, we care more about producing any number of pnl positive strategies than we do the average pnl across generated strategies. more intelligent risk parameterization will be a focus moving forward.
Hypotheses are always just hypotheses, ynkuyk. That said, we saw some awesome things here. And some unexpected that help shed light into where to go next.
Some other smol bugs identified through this process to fix in the shorter term. Then we take some more swings; will share more on this soonly.
Progress every day.
ASYM41b07 continues to move up the curve.
gigabrain trading agent soon.
Yes, yit.
\
Jew Peter retweeted
i am building
a hyperliquid trading agent.
quick context, @ASYM41b07 :
- hypothesizes trading strategies
- turns any hypothesis into executable code
i asked it to generate 100+ trading strategies to set an initial benchmark for its performance.
Here’s what happened:
- 100 of the 140 strategies passed code generation and QA processes;
- 98 of the 100 code-successful strategies were backtested successfully;
So our first benchmark is based on 98 strategies being backtested August 1st - August 8th
The results: not great in aggregate but some promising signs —
93% of strategies that were backtested lost money🥲
But, there were 7% that were PNL positive!
are those just flukes??
when I backtested that set of positive PNL strategies across additional periods, average PNL remained positive, indicating they may be able to hold up even through market change.
ASYM41b07 is immature, and expectations here were low, but these results are promising.
GPT 1 was poor across today’s benchmarks but demonstrated that their root concept was viable.
ASYM41b07 is poor across today’s benchmarks but demonstrates that our root concept is viable.
I have a long list of ways to improve ASYM41b07 in ways that I think will meaningfully increase the quality of its trading success.
There is a lot to dig into with this benchmark run, the strategies that were generated, their backtesting results, and more. Will take some time to investigate the results further and prioritize roadmap from here.
BRICK BY BRICK MFERS
YEAAAAH YEEEEET
\
Jew Peter retweeted
My quant is as brutally honest as he is tireless. Mungi Johnson coded.
100+ algorithmic trading strategies.
being generated,
at the click of a button.
this will set an initial performance standard for @ASYM41b07
some notes:
1. haven't tested at this scale; smaller scale benchmarking is working as expected but will need to see how system handles the (140 strat) request. if it fails for some reason, i will fix and re-run.
2. i am 100% confident most of these strategies will be ass in terms of performance; brace for a low bar/initial benchmark lol. some portion of strategies will also fail to generate; we'll track that rate as well.
3. this is our first benchmark run to set an eval standard for the overall system; many known gaps; much work to be done; many such cases; but this gives us a starting point to understand how well our system can hypothesize and develop quality trading strategies.
4. $250 USD in $ASYM up for grabs to whoever made it this far in the post: we are generating and testing trading strategies for $BTC, $ETH, $SOL, $HYPE, $PENGU, $WIF, and $FARTCOIN
---- comment below in this format:
"
I think @ASYM41b07 will trade $[ASSET] the best
[your name] johnson
\
"
tomorrow, assuming I have the results w/ no issues, I will ask @grok to choose a winner amongst the people who commented with the right asset.
---------------------------------------
glhf
mungi johnson
\
Jew Peter retweeted
WE ARE SO BACK.
ASYM AI can now generate AND backtest hyperliquid trading strategies… in 5-10 minutes.
The new backtesting engine:
- can analyze 1000+ trades in under 30 seconds
- provides penny-accurate P&L reconciliation
- complete audit trail for every trade decision
- w/ 100% deterministic results
Weeks to months of manual work for quant funds,
compressed into minutes.
————————————————————————
additional context & follow ups:
1.
For those just joining: this is a secondary part of a broader trading system: the broader system tracks tokens on Hyperliquid and leverages an LLM pipeline to hypothesize and develop trading strategies——— ;
A + B is much more interesting than B alone: we can generate infinite strategies & filter them down via our backtesting engine. Very cool!
2.
The backtesting engine had been causing a ton of issues; when I first started to develop the backtesting engine, I made 2 big architectural mistakes:
first, I mirrored the live trading system almost 1:1, including setting up mock broadcasting services and mock execution services as external services that a backtesting simulation could connect to in order to replicate the live trading system’s network constraints; this worked but backtests either took too long or ran into bottleneck issues via those exact network constraints; I decided then to move away from that approach but still built on top of a lot of the infra that I had set up at that point, which became very messy, tens of thousands of lines of unnecessary code, etc.
second, I had wanted the LLM pipeline to produce a single strategy class that was environment aware and capable of working within the backtesting environment or the live trading environment; maybe someone else will figure out how to do that, but I struggled, only able to get it to work properly in one or the other, I hypothesize because the environments were different enough that it was difficult for the LLM pipeline to build code that was environment aware, and paired with my then-disgusting-frankenstein backtesting environment, things were brutal to debug.
so I capitulated.
I took the learnings from the first pass at the backtesting engine and rebuilt it from scratch, creating a nice clean and easy to work with backtesting environment.
I built a secondary LLM pipeline that could reproduce the trading logic and constraints perfectly but that would produce code that could work in the backtesting environment.
and that’s what it took to get this working.
3.
Right now I can tell you with confidence: the system would not deploy many strategies, as most are definitively losing strategies atm, but a) we can filter those out, as that’s what the backtesting engine is for, and b) we can now measure the overall system’s ability to trade and we can see how that changes as we add new features and improve the broader system.
I will share more on this benchmarking process s00n along with other details.
mungi johnson
\
Jew Peter retweeted
Make sure to join the $hiki community
The best way to do nothing, is to do nothing together
Hnq9gGF2HFShcNg7U6pU1xxNs7zwvD5E1LoLdjMApump
teapawt.net/i/communities/19581679…
Jew Peter retweeted
bug fixlets & smol improvements:
1. added dynamic Take Profit, Stop Loss, and Time Based exits per strategy, per asset, based on historical data, moving away from a more hardcoded approach
2. fixed an issue with execution simulation by removing number rounding that was leading to imprecise sims
3. adjusted strategy base class to close positions locally to make sure they're in sync with the execution service
4. stripped out unnecessary exit logic from strategy template that was causing live trading positions to close out instantly
5. implemented but not tested: brought live execution simulation to parity w/ backtesting simulation to properly account for volume-based fee discounts, funding rates, slippage & position sizing
still a some more cleanup to do, and then a lot of core work to do from there, but we progress.
\
Jew Peter retweeted
Proof
of
Work.
-----------------------------------------
random:
- the thing i missed more than anything else about working in an office: whiteboarding. buying a massive whiteboard for my house has been liberating.
- lots of alpha here for anyone building an AI trading system (& some garbage too)
-----------------------------------------
asym:
- resolving a couple of bugs atm, should be getting into some more interesting things this week :)
-----------------------------------------
ily
\
Jew Peter retweeted
on AI Trading Strategies:
PRO: generate infinite strategies
CON: generate near infinite losing strategies
quant funds face a similar issue: most hypothesized trading strategies are not actually going to be profitable.
Backtesting (partially) fixes this.
ASYM AI can hypothesize trading strategies, develop code for those strategies, and run those strategies, but without a filtering mechanism, we'd end up deploying more losing strategies than not, and we'd end up losing money.
& so we built a backtesting engine.
The backtesting engine simulates strategy execution for a defined period of time. It:
- uses historical data
- accounts for slippage, spread, & fees
- accounts for Hyperliquid funding costs
- logs entry/exit prices, hold times & P&L for all trades
- evaluates P&L, win rate, drawdown, Sharpe ratio+ for the strategy
& ultimately determines if a strategy should be deployed to the next step or not.
\
Jew Peter retweeted
Replying to @NicoJRNIX @gemxbt_agent
ai agents starting to move again. $asym showing signs
Jew Peter retweeted
Trading shitters is fun!
But very market dependent.
& so we moved our trading system away from Pump and to Hyperliquid.
In June, we began tracking 7 assets for trading:
BTC. ETH. SOL. HYPE.
FARTCOIN. PENGU. WIF.
\
Jew Peter retweeted
this crypto shit sometime be confusin as hell on god these white boys talkin 'bout FUD more like Fuck U Doin nigga 😂




























