chief architect @turbopuffer, previously distributed systems @CockroachDB
New York, NY
Joined April 2013
- Tweets104
- Following135
- Followers1.1K
- Likes363
Nathan VanBenschoten retweeted
turbopuffer crossed $100M run-rate in March. 19mo after $1M. Profitable & <$1M raised.
Cursor・Anthropic・Notion・Cognition・Harvey・Bridgewater・Ramp・Linear・Legora・Superhuman・Atlassian・Granola
We’d be nowhere without them. We work like hell to exceed their expectations.
Nathan VanBenschoten retweeted
SID-1 is an agentic search model by @SID_AI
→ 1.9x recall over RAG + rerank
→ 24x faster, 99% cheaper than GPT-5.1
trained using large-scale RL on turbopuffer at 1k+ QPS bursts over 10M+ document corpora across thousands of steps
tpuf.link/sid-1
We've been pushing search to uncomfortable scales @turbopuffer: 100B vector indexes
To get there, we had to rethink how search algos map onto hardware. The key insight was balancing bandwidth demands across the memory hierarchy
I wrote about the approach and the math behind it
tpuf ANN v3, for when you need to index the entire web
100B+ vectors @ 50ms p50 / 200ms p99 latency
blog: tpuf.link/ann-v3
Nathan VanBenschoten retweeted
tpuf ANN v3 can search 100 billion vectors with a p99 of 200ms
simplicity scales
[v3 in beta, unfiltered search, 1024D, k=10, 92% recall]
Compare-and-swap has an infinite consensus number. It’s universal – powerful enough to solve any consensus problem.
That’s why I’m excited about what we built here.
It’s compare-and-swap all the way down.
Nathan VanBenschoten retweeted
we are looking for a p99 engineer to work on infra/autoscaling/k8s tooling for the 10s of clusters (soon 100s) we run across all 3 clouds.
dm me
Nathan VanBenschoten retweeted
turbopuffer is generally available. petabytes in prod.
turbopuffer.com/
Nathan VanBenschoten retweeted
modern NVMe drives are blazing fast, but software needs to be architected from the ground up to take full advantage of them
we recently rolled out a new I/O scheduler that uses Direct I/O to maximize performance of our SSD caches, avoiding page cache cpu and locking overhead
Week one hacking on search @turbopuffer and I’m shocked: the write:read ratio is way higher than OLTP.
Ingesting and indexing a constant stream of massive batches is heavy on writes. Reads are less frequent—they just need to be fast.
Nathan VanBenschoten retweeted
last week team tpuf welcomed two more p99 database engineers @natevanben and @danhhz ❤️🐡💨
Sign you’ve spent too much time recently with weak iso levels: when trying to recall why consensus requires two durable writes, the prepare phase of paxos starts to look suspiciously like an optimistic, quorum-replicated SELECT FOR UPDATE lock to avoid lost updates (break safety)
This is even more apparent in CASPaxos (arxiv.org/pdf/1802.07000) from @rystsov.
Nathan VanBenschoten retweeted
Ever wanna just know if a line of code is being hit? Or how often? Or from where? In prod!?
We've been hard at work making that just a click away with side-eye.io/!
Nathan VanBenschoten retweeted
I couldn't be more proud of the many teams that contributed to making possible this incredible testament of @DoorDash's success migrating from Postgres to @CockroachDB.
Incredible read that highlights so much database goodness and learnings!
doordash.engineering/2023/02…
Nathan VanBenschoten retweeted
Long-awaited #golang soft memory limit:
The core feature has landed, but I still need to land a few new metrics to help support visibility into this. by @mknyswe
github.com/golang/go/issues/…
Great thread. Geo latency necessitates a reimagining of infrastructure. Compromising on survivability/correctness of state isn't the answer. My team @CockroachDB spent 2 years addressing this head-on by letting apps express access locality in their schema cockroachlabs.com/blog/deplo…
We actually just found out that our paper describing the work was accepted to the SIGMOD'22 industry track!
The next step is end-to-end low-latency serverless apps spread across the globe. One combination I'm excited about is @vercel + @prisma + @cockroachdb