Peter Morgan (@shortishly) profile picture

Founder https://teapawt.net/t.co/eZYHbzBmT5

Harrogate, England
Joined April 2009
Photo by Peter Morgan (@shortishly)Photo by Peter Morgan (@shortishly)Photo by Peter Morgan (@shortishly)
Caching a 🔥regex, split (not copy) a binary and a fast CRC. The path to: 48MB/sec of Kafka produce requests with SQLite and 20MB RSS
1
1
1
151
Tansu can proxy 600k msg/s of Kafka produce requests with a p99 of 1ms and just 13 MB of memory (resident set size)
3
6
1
34
20,942
Tansu surpassed 1k GitHub ⭐ over the w/e! I recently tuned the server writing to null storage. On my Mac Mini M4, Tansu processes 585MB/s using just 12MB of RSS memory! 🔥A Rust stateless Kafka broker with PostgreSQL, SQLite, S3 and in-memory storage. blog.tansu.io/articles/perfo…
1
146
I spoke about just using Postgres instead of Kafka the other day. Here's an Apache-licensed stateless Kafka proxy with a pluggable Postgres (or SQLite, or S3) backend. What stops one from bundling this into Postrges via an extension? It could expose SQL-native functions to use as a pub-sub, and it could expose a true Kafka TCP server (with a background worker). That way it'd remain Kafka wire compatible from within the db machine. And both would share the same tables. I'm not an expert on C nor Postgres extensions, but ChatGPT seems to claim it's doable without much work.
11
17
2
175
17,246
Peter Morgan retweeted
Say goodbye to cache invalidation with PGEC! 🚀 Check out this article on how PGEC uses logical streaming replication to ensure cache consistency and update in real time. 🤖💻 #caching #databases #postgresql #PGEC #realtime #programming shortishly.com/blog/cache-co…
2
2
183