Trata (YC W25) (@trytrata) header
Trata (YC W25) (@trytrata) profile picture

We host conversations that drive public markets

San Francisco, CA
Joined February 2025
The crux of Turn-Credit comes from the Per-Turn judge prompt. We used Codex to iteratively tune the judge prompt to make sure the credits are assigned as intended and to avoid a skewed credit distribution. We finalized the prompt after 3-4 iterations, and it helped overcome the entropy collapse problem by downweighting redundant turns. We wanted to make sure redundant turns are penalized even in successful rollouts. One can think of the Turn-Credit LLM judge as a proxy for a value function or for how a human would assign credit to the turns.
1
4
413
We penalized redundant turns and avoided entropy collapse compared to outcome-only GRPO. We call this method “Turn-Credit”. Our model (Muse-Glimmer-Hedge) was trained with Turn-Credit for only 21 steps and outperformed Muse Glimmer while using 47% fewer tokens. We see a step change across the test time scaling curve.
1
8
651
Today, Trata is excited to share the revised Hedge-Bench: our benchmark tying open-ending reasoning to prediction capabilities. We constructed 92 real-world environments derived from hedge fund analysts’ reasoning traces. Agents reason from frozen information snapshots, and their forecasts are evaluated against future real-world outcomes. No model scores above 50% (GPT-6 Astra being the top performer at 44.2%). For frontier, state-of-the-art models, these results highlight a persistent and wide gap between producing plausible-sounding reasoning and making accurate, economically valuable forecasts. In a semi-verifiable domain like finance, where outcomes are delayed and reasoning is difficult to verify, intermediate expert reasoning traces are crucial. We post-trained @Meta's Muse-Glimmer-30B on 165 environments with per-turn credit assignment using an LLM-as-a-Judge. We observe evidence of generalization: held-out forecasting accuracy at high effort improved by 10.06 percentage points, while out-of-distribution evaluation scores improved by 3.94 points on @OpenAI's GDPval, 2.29 on @Surge’s GDP.pdf, 7.07 on @Harvey’s LAB, and 4.96 on @Samaya_AI’s FrontierFinance under their respective metrics.
13
12
3
92
11,999
Fable 5 (@AnthropicAI) scores 22% and tops the Hedge-Bench leaderboard. Running Fable was roughly 2X more expensive than Opus 4.8 per trial. For an industry where accuracy is mission critical, human judgement isn't going away
2
5
30
3,840
Today, Trata is excited to present Hedge-Bench, the world’s first benchmark focused on evaluating open-ended reasoning in the finance domain. We curated 102 tasks derived explicitly from the reasoning traces of professional hedge fund analysts working with relevant information sources. Most benchmarks are geared towards evaluating naturally deterministic tasks (e.g. updating spreadsheets, calculating formulas). The result is a blind spot on the frontier labs’ part to the competency that matters most in the finance industry: reasoning. No frontier model scores above 16% on Hedge-Bench. We observe Claude-Opus-4.8 actually regresses relative to the last generation of Claude-Sonnet. We see hallucination rates go as high as 82% for multi-step reasoning. These implications extend beyond just the finance domain and into other domains where trust in agent reliability is equally critical (e.g. law).
21
29
3
145
31,370
Which models should our users be leveraging for non-trivial reasoning work? Some quick experiments, graph made in 4 minutes with @Lovable
9
1,118
$MDB, Bull/Skeptic 1.20.26 Analyst 2: "I was thinking more of @MongoDB Atlas as the system of record. It's a weird application of the classic economic hoteling cost. If I have data sitting in there, there's a cost to calling an @OpenAI API to compute my embeddings. But if your Voyage thing inside Mongo does it locally for me, that's great. I don't need to worry about my data being used by OpenAI in certain ways, lower compliance, or Mongo not being approved. So that flywheel is reinforced. I don't need to leave the Mongo universe to achieve my outcomes. As a result, Mongo becomes the system of record. Analyst 1: Is this your view that it increases the legitimacy of being a system of record? Analyst 2: It seems that way to me as I think about it. I think that's what they're going for. They're talking about being a NoSQL data platform, emphasizing their important role in data readiness, and acquiring a capability or technology for generating high-quality embeddings within their ecosystem. They've already got great vector data-based tools. So when it comes to agentic AI, which requires vector search and data sitting in vector DBs for inference, you've got that in place. To me, the pieces of the puzzle that you're laying out sound like they're trying to be a system of record in an agentic AI future. That's the bold view. Of course, someone might say, "It's not going to integrate, and it's going to come down like a house of cards.""
2
9
1,545
$BME.L "Alex Russo was a D-tier CEO. Everyone looked great in 2021-2022, but it became clear in November 2024, when I was seriously considering the stock for the first time, that he wasn't running the business with long-term performance in mind. He was making decisions based on his instincts, going by vibes, rather than using data to inform his decision-making. So, pretty negative on Alex. It remains to be seen regarding the new one. He's a private equity CEO, who had mixed success in his position. Part of that was his fault, and the rest was due to external factors. That's one yellow flag, but his prior role was in a fundamentally different business, so I'd be careful to extrapolate his performance there too much. We've done reference checks on him, and it seems generally positive-to-somewhat of a mixed bag."
3
1,159
Trata (@trytrata) is now on FinTwit. We host anonymous buyside to buyside conversations via voice calls. We work with 125+ funds (vs. 10 in March) and receive 25-30 human-to-human calls weekly. We compile these conversations into a library of investment research. EX: Fund 1 is bullish and Fund 2 is bearish on $OPEN, they ask each other questions they want answered surrounding the key debates for anywhere between 30-60 minutes. The average fund on the platform: ~$200-400mm in AUM, 3 investment professionals, fundamental research focused. These funds use tools like Bloomberg and third-party expert networks, and have regular access to management teams. To the qualified lurkers and anon accounts of FinTwit: Request a call on any ticker (bull or bear). If we pair you, we’ll give you: 1. Access to that transcript and any transcripts on that ticker published through the end of 2026 2. Airpods
18
21
15
120
59,017