building @sixtyfourai | prev @helixdb, neurocognitive aging research, data science @ ucsd
sf
Joined June 2021
- Tweets481
- Following225
- Followers190
- Likes3.6K
putt retweeted
Introducing our RECON benchmark in partnership with @hud_evals
RECON is the first benchmark dedicated to the people research needed for due diligence, fraud investigations, and data enrichment
Sixtyfour's XHigh ranked first among the systems evaluated, with +70.6% weighted accuracy
Here’s how the evaluation worked 🧵
putt retweeted
We partnered with @sixtyfourai to release RECON, a benchmark for people research. Agents get a name and a few identifying details, then have to find hard-to-find facts about that person: 514 facts across 140 real people.
Across 22 systems, precision was close (56–86%), but recall wasn't (8–84%). Most misses were unanswered fields, not misidentified information.
Sixtyfour XHigh ranked #1 at 70.6% weighted accuracy, and High ranked #2 at 54.3%.
putt retweeted
We ran Sixtyfour on Exa’s people and company search benchmarks.
82.8% of our people search results matched the criteria. Exa’s published score: 63.3%.
That is roughly 20 more relevant people for every 100 results.
We came out ahead on all six measures in the chart.
ALT Sixtyfour versus Exa’s published scores on Exa’s people and company search benchmarks. People search, 1,400 queries: first result matches 90.4% versus 72.0%; a match in the top ten 96.6% versus 94.5%; precision 82.8% versus 63.3%. Company search, 605 queries: first result matches 86.4% versus 61.8%; a match in the top ten 94.4% versus 94.2%; precision 79.6% versus 65.9%.
living with @georgecurtiss for 6 days has made me want to touch rust and learn about databases again
putt retweeted
Introducing the world's first way to screen for fraud on platforms you shop from every day.
Every marketplace runs on sellers it has never met. Sellers sign up under stolen identities, post fake listings, and slip back under new accounts after a ban. Now our agents catch them automatically, work that used to take manual investigation.
Seller fraud used to slip through. Now it turns red with our agents.
putt retweeted
introducing folk beta
the personal ai that lives in your texts and quietly runs your life.
built for everyone. just text a number.
i've been using using cc/codex the way boris describes it in his blog, and it's so much better than just letting the agents do it's own thing.
i think at a certain point of yolo-ing, you just don't have a mental image of how things flow/work anymore when the codebase gets large. but this could just be a skill issue
boristane.com/blog/how-i-use…
