programmer, optimist. work: @turbopuffer
nyc
Joined November 2019
- Tweets1.6K
- Following290
- Followers2.1K
- Likes4.6K
morgan gallant retweeted
In lieu of flowers, memorial contributions may be made to:
Johns Hopkins Alzheimer’s Disease Research Center Fund
alzresearch.org/giving/
Alzheimer’s Foundation of America Caregiver Resources
alzfdn.org/donate/
Photo Credit: Tommy Garcia
it’s exceptionally not cool to find someone’s personal phone number online and cold call them trying to sell to the company where they work at
two thoughts:
- will make damn well sure tpuf never uses pump.co
- objectively funny company name
morgan gallant retweeted
new: highlighting
extract the text fragments most relevant to a query
→ highlight matches in your search results UI
→ minimize context passed to your LLM
docs (and playground): turbopuffer.com/docs/fts#hig…
latency(tpuf search w/ native embeddings) < latency(fetch embedding) + latency(tpuf search)
makes for some beautiful traces
now in beta: native embeddings in tpuf
embedding is the most painful part of puffing. we want to make it easy
you can now convert chunks to vectors as you read and write to turbopuffer, without extra calls to an embedding model provider API
docs: turbopuffer.com/docs/embeddi…
morgan gallant retweeted
Globe&Mail wrote a generous piece about turbopuffer theglobeandmail.com/business…
TIL: script.google.com lets you run automations on your calendar(s). in this case, running an hourly sync of a personal calendar (only specific events) to your work calendar as "busy" events
excited to get this out there in the hopes that other folks can benefit from our work here! a lot goes into making a good tokenizer
also wanna make sure I thank the tantivy folks, we've built our earlier tokenizers (everything before word_v4) atop their lovely open sourced work
so @morgallant has optimized FTS tokenization throughput to 423 MiB/s and open-sourced it (github.com/turbopuffer/alyze)
I keep telling him that it would be really high agency to get to DRAM bandwidth (~100 GiB/s), and he keeps getting annoyed at me and making it faster
fangirling over being on dash 8 for the first time in a while — i think my first ever flight was on a dash 8!!!
morgan gallant retweeted
BM25 efficiently scores text, but relevance often depends on more than text (recency, popularity, PageRank)
we score non-text attributes as clauses in the same MAXSCORE plan that evaluates BM25 → better first-stage relevance, still scales to 100M+
tpuf.link/rank-by-attr
hoping to write a blog post on this eventually, it's a fascinating topic.. so many different possible approaches, each with their own tradeoffs. for v1, as is always the case at tpuf, we've optimized for simplicity
new in tpuf: regex indexes
regex and glob filters can now use a trigram index to avoid full-table scans
ALT A chart showing p90 query latency performance gains for regex and glob filters after the introduction of a trigram regex index on turbopuffer. Latency improved from 115 milliseconds to 33.8 milliseconds for a representative regex filter and from 267 milliseconds to 34.2 milliseconds for a representative globbing filter. The chart suggests that while these kinds of filters already worked, now they puff!
btw hiring for a systems eng. role at @turbopuffer soon on the text team, you’d work closely w/ @jpountz and I on billion-scale {storage,ranking,query eval}. DMs open!
joint work with @jpountz, covers some of the storage work behind the ftsv2 launch!
for FTS v2, we redesigned our inverted index structure
• tighter compression
• less KV overhead
• better MAXSCORE interaction
up to 10x smaller indexes → up to 20x faster text search!
tpuf.link/fts-index
still in awe of the talent density on the @turbopuffer team… truly such an honour and a privilege to get to work with these folks every day
I really hope that the continued success of Zig will help convince more systems programmers that treating memory allocations as infallible is a terrible idea for PLs, and that OS-level memory overcommits in Linux server environments is similarly terrible!
joint work with @jpountz and @nikhilbenesch; excited to start rolling this out more broadly! insane amount of progress on FTS recently :)
FTS v2: up to 20x faster full-text search
turbopuffer is now on par with Tantivy and Lucene for many queries, more to come
v2 now in beta. 2 PRs away from all query plans being implemented. will be enabled in prod for all, shortly.
ALT Horizontal bar chart comparing turbopuffer FTS v1 vs v2 latencies for five queries on English Wikipedia, v2 much faster (3–20ms) vs v1 (8–174ms).
morgan gallant retweeted
Something that's great with Lucene is how you can run luceneutil benchmarks on a PR to assess the performance impact. So I built something similar for turbopuffer (off the Tantivy benchmark!). Here, baseline is what runs in production now, contender is what will get deployed soon
hand-rolled a fully uax#29-compliant tokenizer today, brain is a little fried but it was lowkey pretty fun
one of my favourite ways to deploy to @Railway! build the binary locally, package it up in a light alpine image and ship it off
