
Madridista @realmadrid | Backend & Systems Engineer
New Delhi
Joined July 2022
- Tweets12.9K
- Following178
- Followers455
- Likes53.9K
std::panic retweeted
Pablo Maffeo was born in Barcelona to an Argentinian mother and came through Manchester City’s academy. This is the worst human being on the planet
std::panic retweeted
Its an weird question, but I'm serious. Let's say 5 minutes to go away at the camp nou. If kylian starts kissing jude, not a single defender will be watching the ball, leaving yan open for a free goal. Would kylian and jude be sent off or would the goal count, thoughts?
This project has been literally built from scratch, no third party dependency at all.
This is the real building from first principles where you have to literally create your own library and tools from scratch to use them in your project.
What this thing teaches you will never actually forget about how the actual rendering, gui, graphics, etc are actually implemented beneath.
There are more to this project which i certainly not aware of but such projects teaches one a lot about the stuff which carries the load beneath.
A great project by @RubenVeidt, it almost took bro one year to get all of the stuff done and lets see what mores coming ahead.
std::panic retweeted
I've been building a tool that visualizes different databases, their use cases, and how they connect to each other
I'm starting to feel like two dimensions aren't enough to capture all the nuances though
open to ideas/suggestions
std::panic retweeted
it's mainly because of the people associated with the protest.
they've invited arundhati roy to the stage, someone who opposed the hanging of afzal guru. she believes that kashmir is not an integral part of india.
there are posters of aisf and sfi all over the protest.
sfi is affiliated to cpi(m), the party that officially stated in its manifesto that they want denuclearization. yes, they literally want india to not possess nuclear weapons. the same nukes for which we're forever thankful to dr homi bhabha, dr apj abdul kalam, and our scientists.
aisf is affiliated to cpi. ignorant kids on insta are blindly adding their posters to their stories, ignoring that this is the same party that opposed the pokhran-ii tests and the abrogation of article 370.
there are so many people associated with the protest who support umar khalid, and you can just check out his statements on kashmir.
you can't claim to care for the nation and shake hands with those whose entire ideology is to destroy its integrity.
sure, there are a lot of genuine people involved in the protest because the cause is genuine. the education minister must resign. the govt is directly responsible for the students who've committed suicide because of this fiasco.
but as a matter of principle, i can't associate myself with a protest that has these elements sharing the same stage.
std::panic retweeted
Replying to @9elcapitano @uniqueash10
Maybe but lets see if they lose or win now all 3 games there will no suprises imo, all are top 5 ranked teams in the world so good semis I would say I didn't want any fucking underdog and shit in the semis
std::panic retweeted
tryna make something similar to a cloudflare workers but cloudlare only support typescript and webassemply i am trying to do similar thing with thier v8 isolates based thingy but making a AOT compilation pipeline to webassembly and then that runs on the isolates
std::panic retweeted
Now implemented a 4x8 AVX2 Microkernel for GEMM, first as unpacked then as packed. At a high level what it does is, instead of processing 1 row at a time, process 4 rows × 8 columns. Instead of loading B parameter fresh for every row, B is loaded once and reused across 4 rows.
At 256×256 matrix size this hits 72.5 GFLOPS, but at 2048×2048 it collapsed to 15 GFLOPS because B access strides across 2048 elements (constant cache misses).
That's where packing comes in: copy tiles of A and B into contiguous buffers before the micro-kernel runs. Now every load is L1 sequential instead of striding through RAM. Result: 69.2 GFLOPS at 2048×2048.
okay, i have so much to talk about.
storage layer for kova is shipped. it's a from-scratch vector DB in rust, and the storage layer is the bit where every decision has to survive a kill -9 and come back the same. that constraint touches every file, every fsync, every byte format
vectors live in an mmap file. fixed stride (a tiny per-slot header plus dim*4 bytes for the data), and every slot self-describes its id and a present flag. no sidecar index file, no separate id->offset map. the cost is that open is O(N): you walk every slot to rebuild the in-memory id_to_slot. the win is there's no two-file atomicity problem: the data IS the index, can't drift from itself. for the workloads i care about (millions of vectors, opened rarely), that's a great trade
file growth is doubling, and never shrinks. doubling so per-insert grows amortise away and do not cost much. never shrinks because truncating an mmap region with an outstanding reference is SIGBUS, exactly the kind of bug i shipped once and found much later. deletes don't shrink the file either; freed slots go on a free-list and the next insert reuses them, so a delete-heavy workload stays compact without anyone running vacuum
metadata is the opposite shape, so it gets opposite treatment. variable-size (open key-value bags), cold (only read for the final k candidates, not on every graph edge), small in aggregate. so the metadata store keeps the whole map in memory and persists via atomic_write (tmp + fsync + rename + dirsync) on mutation. full-file snapshot, no mmap, no sidecar, no free-list. forcing mmap onto variable-size data would mean a separate id->(offset, length) sidecar plus a free-list for resize-on-update, which is a B-tree in disguise. different access patterns deserve different storage strategies; the same hammer for both is just dogma
the WAL is segmented (64 MB rotation), CRC32 + length framing per record, torn-tail recovery on the active segment, multi-segment replay in LSN order. the segmentation makes truncate O(1): delete the superseded files
all of that gets composed by Shard under strict log-then-mutate. every insert moves through three phases: validate, commit (wal.append + wal.sync), apply (index + metadata). phase 3 failures panic. the WAL is truth; pretending otherwise to the caller is the only way to corrupt the log on retry. postgres ships this. rocksdb ships this
the real centerpiece is Shard::checkpoint. on paper it's "snapshot the index, truncate the WAL." in practice it's six phases that have to commit atomically across multiple files: vacuum tombstones, fsync the WAL to capture a durable LSN, stream the HNSW graph to a snapshot file, write the manifest, truncate the WAL, delete the old snapshot. anywhere in those six phases the process can die, and every kill window has to have a defined answer
the whole thing rests on this: the manifest is the only commit point. a tiny atomic_write of a few bytes, names which snapshot generation is live. everything before it is staging. everything after it is best-effort cleanup. kill before the manifest commits, reopen sees the old world and replays the full WAL. kill after, reopen sees the new world and replays WAL only past the checkpoint LSN
the snapshot itself is also atomic_write'd (streaming variant, so we don't buffer the whole graph in memory before the rename), but you can't commit snapshot + manifest as one operation: POSIX has no atomic_rename_n. so the problem splits naturally. write the big data file atomically and fsync it. then write the tiny pointer file atomically. the manifest exists precisely so that multi-file durability has a single commit point above the data, instead of trying to make the data itself self-committing. and keeping them separate means the checkpoint pays for one snapshot rewrite per checkpoint, not per manifest update; the pointer stays cheap to update even when the thing it points at is huge
and snapshots are named graph.{N}.snapshot, not graph.snapshot, for the same reason. a single overwrite has a half-written window where the file is partial but the manifest still points at it. generation numbers let the old file stay valid right up until the manifest atomically swaps the pointer. same trick postgres uses for its SLRU pages. names are cheap, ordering is not
300 SIGKILL torture iterations. 55,697 acked inserts. 962 checkpoints. zero data loss. 234 tests green.




















