Skillsmaxoor. Currently learning math for ml & system design.
Joined March 2018
- Tweets7K
- Following481
- Followers645
- Likes42.5K
Sayid retweeted
Learning resources play a key role in shaping where talent goes. Today I’m launching my side project, BioTorch, in public beta to give researchers a clearer path into biological AI.
LLMs have a wealth of great tutorials, courses, and minimal implementations from great teachers. They make the path from curiosity to contribution easy to see and offer researchers from fields like math or physics a legible roadmap to transition into AI research.
In bio AI, that path is much harder to piece together. That friction can mean fewer researchers entering the field, fewer startups getting built, and less investment allocated to promising ideas. This ultimately leads to fewer life-saving cures being discovered than would otherwise be possible.
BioTorch is my attempt to help change that.
It starts with 116 PyTorch exercises and 17 model guides. You implement the building blocks, test your code, and see how the pieces fit into seminal models like AlphaFold2, ESM2, and RFdiffusion.
The goal is to make influential papers in bio AI something you can understand and build on, bringing researchers up to the frontier as rapidly as possible.
I want more people working on biology’s hardest problems. My mission is to help researchers find their way into the field, and BioTorch is the result of that ambition.
BioTorch is completely free while in beta.
Try your first problem now!
biotorch.chrishayduk.com
Sayid retweeted
Never underestimate the power of habit. Even if an activity is unpleasant right now, it doesn't mean it will always feel that way.
"When I was starting out learning math, I wasn't really sure if that feeling of waking up and not looking forward to starting my day with math was ever going to go away.
But around the 60 day mark, my brain was like "we thought this was a temporary thing, but it doesn't seem like you're going to stop doing this any time soon, so we're going to have to do some rewiring to make sure you're not feeling repulsed, you're not feeling anxious."
And then magically one day these feelings just completely went away.
If you stick with something long enough, then it will just become the new normal for you.
Your brain, your body will eventually be forced to adjust."
-- @sayidislearning
Sayid retweeted
I need to pay you for this prompt!
Give LLM your system prompt and use my personal learning notes as input, Anki cards are created in a minute for me to swipe on the train
Here you go:
Template for Creating Anki Cards
You are helping create Anki flashcards for someone who reviews thousands of cards daily. The cards must eliminate all second-order thinking and friction to enable automaticity.
Core Principles
1. Eliminate Second-Order Thinking
Bad: "What are tasklets? A simpler way to create bottom-half work"
Forces questions: "Simpler than what? How are they used? What's the alternative?"
Good: "What are tasklets? There are 10 softirq slots. Only one handler function can be registered to each softirq slot at a time. If a device registers to a slot and another device wants to access that same slot, it has to handle coordination/access logic.
Tasklets are a way for devices to create a "tasklet" datastructure and drop it off at the handler function & let it process it for you."
No follow-up questions needed
2. Make Cards Follow Up Proof
Each answer must pass the "interview follow-up test" - if someone could reasonably ask "but why?" or "how?" after your answer, the card needs more detail.
Bad: "Why does interrupt coalescing improve performance? Reduces overhead"
Interviewer asks: "What overhead? How does it reduce it?"
Good: "Why does interrupt coalescing improve performance? One handler entry/exit cycle processes multiple interrupts instead of separate cycles for each interrupt"
Complete mechanism explained, no follow-up needed
3. Use Natural Language, Not Academic Language
Bad: "CPU-generated interrupts triggered by illegal operations" Good: "CPU-generated interrupts triggered by invalid operations like dividing by 0"
Bad: "Processes in uninterruptible sleep state awaiting pending I/O operations" Good: "Processes go into uninterruptible sleep state waiting for slow I/O operations to finish"
4. Match Question and Answer Specificity
Bad: Generic question with specific answer:
Q: "What's the performance benefit of interrupt bottom half processing?"
A: "Bottom half can process multiple network packets in one function call"
Good: Either make both generic or both specific:
Q: "What's the performance benefit of interrupt bottom half processing?"
A: "Multiple similar operations can be batched into fewer function calls"
5. Avoid Vague Comparative Language
Bad: "Easier", "smarter", "better", "more efficient" without context Good: Concrete technical mechanisms
Bad: "NAPI is a smarter way to handle network traffic" Good: "NAPI switches from interrupts to polling when packet arrival rate becomes too high"
6. Eliminate Abstract Terms That Create Follow-Up Questions
Bad: "High-performance subsystems" → Good: "Networking and block I/O" Bad: "Critical sections" → Good: "Shared data while it's being modified" Bad: "CPU cores become bottlenecks" → Good: "CPU cores hit 100% usage" Bad: "Overhead" → Good: "Handler entry/exit cycles"
7. State Complete Cause-Effect Relationships
Bad: "Hypervisor processes hardware interrupts then delivers virtual interrupts to guest VMs"
You have to infer: "Oh, so it processes twice, that's why it's higher overhead"
Good: "Each hardware interrupt gets processed twice - once by hypervisor, once by guest VM - doubling the handler entry/exit cycles compared to bare metal"
Complete logical chain included
Quality Checks
Before finalizing a card, ask:
Second-order thinking test: Does this answer require me to make any inferences?
Interview test: Could someone reasonably ask "but why?" or "how?" after this answer?
Natural language test: Does this sound like how people actually talk?
Specificity test: Do the question and answer match in detail level?
Atomic fact test: Is this testing exactly one concept that can't be decomposed further?
Common Fixes
Replace "overhead" with specific operations (context switches, handler entry/exit)
Replace "bottleneck" with concrete measurements (100% CPU usage)
Replace "critical sections" with specific scenarios (shared data modification)
Replace "race conditions" with actual consequences (data corruption)
Replace "high-performance" with specific systems (networking, block I/O)
Replace "consider switching" with concrete actions (time slice expired)
Replace academic terms with natural speech ("illegal" → "invalid")
Output Format
Question[TAB]Answer[TAB]Tag
Example of Progression
Original: "What are work queues? Bottom-half work handed off to kernel worker threads that can sleep"
Problem: "Handed off" is vague, "that can sleep" makes you ask "why does sleeping matter?"
Fixed: "Bottom-half handlers executed by kernel worker threads that can sleep and block unlike other interrupt processing"
Sayid retweeted
ITS LIVE. FINALLY!
Intro to Quantum Computing
This has been 8 months in the making.
Mastery learning style, built in spaced repetition, well scaffolded, etc.
The beginning of "Math Academy for Physics".
(no affiliation, and a bold claim, I know, but hear me out):
Sayid retweeted
This post is for men only.
30 years ago, age 24, was the first time anything every fully synthesized for me. Perfect spring day, on my job site, I looked up at this blooming tree. Indelible memory. Sitting with the foundation guys at lunch break. I finally felt euphoria of “it” happening. The one moment, my private élan vital, where my future was finally happening. “It’s working.” Not some hope or talk but the real deal. And I want it again. Huge driver. I’ll get it. Isn’t some “our team won” high. A massive firmware update.
Dear reader, some might know this feeling. I know most don’t. Men today are being shamed into a corrosive belief that they are defective, need rehab, should move over, submit to punishment for forefathers sins for whom history has been neatly rewritten to inspire guilt and submission. Anyone pushing this is trying to take something away from you. Your power to be the powerful man you are, never to get your moment that defines us.
In Aug I read Meditations by Marcus Aurelius. Honor, honesty, duty, responsibility. Our code, the crucible of good men.
Storms arrive, and you’re expected to be the lighthouse. You either shine bright and perform, or you let the boats sink around you. Becoming a hero is a choice.
I’m always working to rise to the top. It’s there where I’ll earn this next-level sensation, and challenge myself to create more distance from the weakness I observe. It’s my duty to become a Monumental Man.
Sayid retweeted
"schools were never optimized for learning"
Sad but true. The field of education made absolutely no sense to me until I realized this. And then it made perfect sense.
So many debates aren't about whether techniques X, Y, Z are effective learning techniques. They're about whether education should seek to maximize learning.
Lots of people in education disagree with the premise of maximizing learning.
-- There are plenty of students who would prefer for their education to maximize other things like fun and entertainment while, as a secondary concern, meeting some low bar for shallowly learning some surface-level basic skills.
-- And there are plenty of teachers who are incentivized to use easy, fun, low-accountability, hard-to-measure practice techniques that keep students, parents, and administrators off their back. (Unfortunately, these practice techniques tend to be ineffective.)
The subfield within education that seeks to maximize learning is known as “talent development.”
In talent development, the optimization problem is clear: an individual’s performance is to be maximized, so the methods used during practice are those that most efficiently convert effort into performance improvements.
Practitioners of talent development tend to be found in hierarchical skill domains like sports and music, where each advanced skill requires many simpler skills to be applied in complex ways. This is because it’s hard to climb up the skill hierarchy without intentionally trying to do so.
To learn an advanced skill, you must be able to comfortably execute its prerequisite skills, and the prerequisite skills underlying those, and so on. Getting to the point of comfortable execution on any skill takes lots of practice over time – and even after you get there, you have to continue practicing to maintain your ability.
None of this happens naturally. If you don’t carefully manage the process, then you struggle. Nobody gets to be really good at a sport or instrument without taking their talent development seriously and intentionally trying to maximize their learning.
Now here's why talent development is so necessary in math.
Success in sports and music requires talent development. But most students aren’t expected to achieve a high level of success in sports or music, so they can get away with de-prioritizing talent development. If every student in gym class were expected to be able to do a backflip by the end of the year, things would have to change – but the expectations are so low that meeting them does not require talent development.
When it comes to math, things become problematic. Like sports and music, math is an extremely hierarchical skill domain, so achieving a high level of success requires a dedication to talent development. However, unlike sports and music, most students are expected to achieve a relatively high level of success in math: many years of courses increasing in difficulty, culminating in at least algebra, typically pre-calculus, often calculus, and sometimes even higher than that.
As a result, in math, de-prioritizing talent development leads to major issues. When students do the mathematical equivalent of playing kickball during class, and then are expected to do the mathematical equivalent of a backflip at the end of the year, it’s easy to see how struggle and general negative feelings can arise.
Sayid retweeted
From Sudan to Silicon Valley: Beyond the Resume
link.medium.com/A2UhOn726Nb
Sayid retweeted
My greatest accomplishment in life has nothing to do with the work I've done.
It was successfully raising my son to be a better man than me.
When he was born, I only knew one thing: I wasn't going to be my father and neglect my child.
Sayid retweeted
My mom got emergency surgery for colon cancer almost three years ago. Stage 3A.
Since then, it's spread to her lungs, stomach, and lymph nodes, and her diagnosis was moved to stage 4. She's 50, raising two teenage sons.
My grandmother just told me her condition is worsening rapidly. While she's been working as a schoolteacher -- often while wearing a chemo bag -- lately she's been missing work more and more. She's out of sick days and out of vacation days.
Barring a miracle, things are about to change in the next several months.
Early on in this, when I was still in the military, my command moved me to NYC, where she chose to live, and I took her to chemo every week. Later, when her condition was stable, she said to me: "Live your life. Do whatever it takes to succeed. Go travel, write, live. Don't sit around here waiting for me to die."
I took her advice. Nevertheless, everything I have done in the last two years has taken place under the shadow of my mother's early death. This takes a certain amount of blithe optimism that can border on compartmentalization. I just don't think about it, not out of callousness, but because -- what can I do?
Everywhere I travel, I know I could get "the call." I know by then it'd be too late to say goodbye. Whenever I go to Utica and get the train elsewhere, I know I could be nailing it down that I never get to say goodbye to my mother. That maybe I miss her funeral. That my kid brothers will wonder where the hell I am as they start to figure out life without our mom.
Now, she's lost her hair. She's on new chemo with vicious side-effects. She's faced with no longer working, and with moving her teenage sons upstate to move in with our grandmother. She's faced with accepting an untimely end to all of her dreams -- with contending with the weird waiting room between "life as she wishes to live it" and "death."
To get to that dream, she got clean from hard drugs years ago. She fought to be near me, her estranged son. She divorced an abusive pill addict. She put herself through college, then through grad school, then through grad school again. She packed a U-Haul and moved to Brooklyn, raising two young sons single-handed, and working 60, 70, 80 hour weeks, running on no sleep.
She did it -- she crawled up from the bottom and made it as a public school teacher in NYC. She fought every conceivable hurdle and lived her dream.
Less than a year later, she got the diagnosis -- the death sentence.
Please pray for her -- for Rachel. Pray for my brothers, my grandmother, her husband, for everyone around her. Everyone's carrying around a lot more than they realize here. This shit is truly awful, and particularly for the kids -- absolutely frightening.
Sayid retweeted
You could live in Utica.
$850/mo apts, city views, fish frys, chicken riggies, Utica Club, classical architecture, Amtrak to NYC in 4.5hrs. Earning $40k/yr managing the deli dept at Price Chopper.
No stress -- a beautiful life.
But you don't want it. People want stress.
Sayid retweeted
I want more children. If we hit three and I get comments I’ll just say “Yeah, I’m taking over the town.”
Sayid retweeted
When I completed my California teaching credential from 2020-21 and attended numerous professional developments (PDs) from 2019-23, the science of learning was strangely absent, almost like it was taboo.
Instead, the credential and PDs were 100% focused on diversity, equity, & inclusion: readings (and an essay) on hegemonic heteronormativity, anti-racism training, “sharing circle” training, and even a presentation on the gender unicorn, complete with an extraordinarily complex gender classification flowchart, just to name a few examples.
Forget the science of learning – even the most obvious practical skills that a teacher would need to exercise on a daily basis, such as managing a rowdy classroom, communicating with parents, holding students accountable for their work, and dealing with academic dishonesty, were not covered at all in teacher credentialing nor PD.
However, there was no shortage of pointless activities.
I vividly remember a virtual PD in which the first 30 minutes was spent going around the room of 30+ teachers, the PD leader asking each teacher to describe the weather outside in their physical location and explaining how their personal feelings that day related to the weather.
At another PD, there was an activity involving a circle of traffic cones, each cone with a different emoji taped to the top. The PD leader read through a list of words and asked the teachers to walk over to the cone that they felt matched their feelings in response to the word.
Sayid retweeted
At the end of the day you can either waste time debating your coach on the training regimen, or you can use that time to just put your head down and do some f*cking work.
Sayid retweeted
How do you increase a student's ability to make mental leaps?
Perhaps counterintuitively, it’s not by actually increasing their mental “jumping” ability.
Instead, it’s by helping them build knowledge “bridges” that reduce the distance to jump.
Sayid retweeted
Developing hardcore skills is probably the biggest bottleneck to improving one's life and society in general. Doesn't which of those things (yourself vs society in general) you're more focused on -- hardcore skills is always the answer.
So many people want to have high impact and improve the world (and their own lives) in a big way, but desire is not enough, you typically can't do anything big unless you have big skills.
I say "typically" because sure, some people get really lucky being born into the right family in the right place at the right time and enjoy an outsized impact despite not having built up their skills as much... but you know, even for those people, the difference between a relatively large impact (relative to other people) vs an absolutely large impact ("put a dent in the universe") still comes down to skill-building.
Hardcore skill development is also one of the greatest social mobility hacks. Even if your family is not well-connected, you can make up for it by developing real skills.
Sure, you have to develop more skills than well-connected people to reach the same level of opportunity, and you're going to have less guidance developing those skills and finding your way to the arena -- but once you're in the arena, those extra skills pay big dividends.
Personally, learning advanced math at a young age completely changed my life trajectory. I definitely had a more fortunate/advantaged childhood than the average person, but at the same time, there is not a single person in my family who is mathematical, technical, scientific, or academic.
High-impact career guidance was completely absent. (For instance, the sole contribution of my HS counselor to my college apps was misspelling the word "Calculus" as "Calculas" on my transcript.) If I hadn't skilled up on math & quantitative coding in a hardcore way, there's no way I would have landed anywhere near the stuff I'm working on now.
Another crucial thing that hardcore skills did for me personally was give me the ability/confidence to take more risks knowing that I'd be okay. I leveraged those skills into getting a data science job in college that paid enough for me to build a little financial safety net by the time I graduated, and that safety net allowed me to move out to a big city and take on scattered work in my area of passion, looking for the right opportunity while spinning my wheels in a state of cashflow neutral.
There's no way I would have found a great career fit without that period of exploration, there's no way I would have had that period of exploration without the safety net & confidence that I could get a decent job if I depleted it, and the root cause responsible for all that was hardcore math / quantitative coding skills.
Sayid retweeted
There's this fella on here that built a pretty successful brand around his weight loss journey.
He was posting updates daily on his workouts/nutrition, going viral at least once a week and I think he was one of the first "losing weight in public" type accounts to take off on Twitter.
He lost my respect and ruined his reputation once he got himself caught up in a web of lies after it became clear that he wasn't losing weight at the rate he was claiming to be losing.
I think there's a lesson in there for the folks like myself who are learning in public.
The biggest downside to learning in public is not having anything to show for it.
Everyone loves investing in underdogs, but that investment comes with the expectation that you're going to fall through with whatever it is that you said you were going to do.
However, if you do end up sticking to your word and have some measurable output that people can rally around and congratulate you for, then there's only infinite upside.
Your work ethic becomes publicly accessible for anyone to validate.
I bootstrapped my tech career from working at a warehouse while taking care of my my mom and younger siblings, so unless a baddie falls in love in with me (in my future arc rn so that shouldn't happen) or I fall ill or god forbid die, there's no scenario where I flop.
I didn't learn about tech twitter and the whole learn in public thing until I already started my tech career but now that I want to pivot into ml/ai, might as well share what I'm doing with the world since I'm already doing the work.
To conclude my yappathon, learn in public at your own risk.
FYI @kelseyhightower was an early investor (who is also an advocate for learning in public) & he got his ROI :)
Sayid retweeted
As an educator, I agree with much of what he has said, but I also have some reservations.
There is a significant difference between "edutainment" and authentic learning, and I agree with that distinction. There’s nothing wrong with indulging in some edutainment, but it’s crucial to differentiate between the two.
One issue with edutainment is that people often *think* they’re learning when they’re not. They watch an edutainment video and say, “Oh yeah, I totally get that.” However, those same people can be completely stuck when presented with a problem that requires the knowledge from the video. In other words, they haven’t learned anything! Real learning occurs when actively solving problems, not when watching videos.
So he’s right: You should ask, “Is this edutainment, or is this real learning?” If it’s edutainment and your goal is to learn, find another resource. This principle works both ways.
My colleague @justinskycak recently wrote about differentiating between edutainment and authentic learning.
xitter.cf/justinskycak/status/18…
Real learning should feel like a solid workout with an experienced personal trainer who pushes you beyond your comfort zone so you can make tangible progress. Just like press-ups are not “designed” to be fun, learning tasks should also not be designed for fun. The purpose of a learning task is to force a positive change in long-term memory so it stays there. “Fun” doesn’t factor into it.
However, a highly efficient learning session can be fun because you’re making solid gains and feel great because you can sense your progress. Have you ever been to the gym and had an incredible session where you smashed many records in a single day? Was that not fun? You may not have sought out fun, and the routines were not “designed” to be fun, but it became fun because you made incredible progress and loved the feeling.
In other words, fun should be a by-product of an incredible learning experience where students are almost in disbelief at how much they’ve achieved. But when designing learning tasks themselves, it’s best to keep “fun” out of it.
Regarding the point, “Don’t just read; take notes, re-read, re-phrase, process, manipulate, learn,” I agree with one addition. The goal isn’t to take notes verbatim and constantly use them as a crutch. Ultimately, you want the knowledge in your brain, not on paper. Notetaking, rephrasing, etc., is a means to an end, not an end in itself. You should continuously try to wean yourself off written notes until they’re no longer needed.
xitter.cf/ninja_maths/status/181…
The “re-phrase, process…” part involves checking your learning. I wrote about this here:
xitter.cf/ninja_maths/status/181….
When it comes to anything involving heavy-duty math, I think it’s misguided to suggest that learning should always be done top-down, depth-wise. This is *especially* true if your math foundations are weak to begin with.
The top-down, depth-wise approach works if you’ve mastered most (preferably all) of the foundational prerequisites needed to solve a particular problem. When you’re experienced enough, there is a certain amount of “filling in the gaps” you can do as you progress. However, if the gaps are vast (i.e., if you haven’t even grasped basic high-school math, yet you’re attempting to tackle a problem that requires graduate-level math), you’re unlikely to succeed because you lack the foundations. “Drilling down” to acquire the necessary math is a strategy doomed to failure.
Some people are exceptionally bright, and Karpathy is one of those people. But for most of us, a top-down approach to a problem involving heavy-duty math will likely lead to significant confusion, frustration, and eventual giving up. Those individuals need to step back, work on their foundations, and build up to the math required for the problem they’re trying to solve.
Hell, I even struggled with this personally when I was a PhD candidate, and my prerequisites (a first-class degree and a master’s) were pretty solid!
xitter.cf/ninja_maths/status/182…
I’m in total agreement with points 2 and 3.
“Bottom-up” or “top-down”? What’s the best approach to learning math? I can answer this from experience.
My biggest mistake when starting my doctoral research was taking a “top-down” approach.
I focused my efforts on a handful of research papers on the frontier of my chosen field, even writing code to solve problems in these papers from day one.
However, I soon realized I lacked many foundational prerequisites, making the first year exceptionally tough.
What I should have done was spend 3-6 months dissecting the hell out of all the key research papers and books written on the subject, starting from the very basics (from my knowledge frontier) and working my way up (the bottom-up approach).
I eventually abandoned the “top-down” approach, solidified my foundations, and made significant progress in my research.
I get the impression that many machine learning students are taking the “top-down” approach and wondering why they’re getting nowhere with the math. If this sounds familiar, you could be making the same mistake I did.
Spend a good chunk of time mastering single-variable and multivariable calculus, linear algebra, probability, and statistics. This will give you the best chance of mastering this fascinating and challenging field.
Your future self will thank you.
Sayid retweeted
not one piece but:
- put God first in your life
- embrace the biblical moral axioms
- having a wife and kid speed runs your character development
- choosing a wife is the most important decision you will ever make
- design your life around maximizing peace
- learn what sacrifice means
- drop p*rn and minimize video games
- read the naval quote on what to learn
- lock in and get cracked
