#CS Associate Prof @YorkUniversity #ComputerVision Researcher, @VectorInst Affiliate @UofT status only, @ELLISforEurope Member, X-social media chair 4all vision

Toronto, Ontario
Joined August 2015
I’ve launched 2 new course websites for my Introduction to #ComputerVision and #DeepLearning in Computer Vision courses. 🆕 Gaussian Splatting 🆕 Flow Matching 🆕 Sparse Autoencoders 🆕 Lecture Notes (work in progress) The videos don’t include voiceovers, but that’s coming in a future revision 🤞 Share your feedback in the thread.
8
31
1
324
44,943
Kosta Derpanis retweeted
Proud of what the Ideogram team built. So many possibilities when you combine design taste, Ideogram Vector, and coding agents. 🎨🚀
Today, we’re introducing Ideogram's first vector model. Generate clean, editable SVGs with smooth gradients and sharp typography from a text prompt, or turn any image into a detailed, scalable vector. Live in Ideogram and the API: ideogram.ai/models/vector
11
9
76
3,228
Kosta Derpanis retweeted
If you have ever trained a Transformer Language Model and wondered why there are "spikes" in loss -- our recent paper (arxiv.org/abs/2610.13524) gives an explanation that arguably gets to the most essence so far, as it replicates loss spikes by simple quadratic models.
4
16
1
163
8,456
Kosta Derpanis retweeted
1/ JEPA selects which features its world model keeps. But how? Will it keep a feature it can't predict, like a traffic light? Judging each feature alone, earlier work says no. We show evidence that this is wrong and introduce a new characterization that explains why. 👇🧵
3
22
6
142
16,593
Kosta Derpanis retweeted
🚀 Are you a PhD student pushing the boundaries of AI, robotics, graphics, or accelerated computing? NVIDIA's Graduate Fellowship offers up to $60K in funding, mentorship from NVIDIA researchers, and a summer research internship! ⏳ Apply by October 30! blogs.nvidia.com/blog/applic…
4
9
65
6,984
Kosta Derpanis retweeted
Yesterday a junior researcher messaged me, frustrated and a little scared. They had just read that a company pretrained its model with a single researcher. One person for the entire pretraining. no R&D team, no nothing. If one person and a cluster can do what used to take a whole team, then the expertise they had spent years building is already obsolete. I get the fear. Marketing headlines like this are built to produce it. But I think the conclusion is wrong, and the reasons it is wrong say a lot about what a researcher is for now. Here is what I told them, expanded: 1. Be adaptive: RSI is real, but this isn't it yet Recursive self-improvement is real, and you should plan around it. In September, OpenAI said its research org now logs 3.1 agent-workdays per human workday, and it aims for an automated AI researcher by March 2028. Models may soon run most of pre- and post-training. If your five-year plan assumes otherwise, change the plan. We have already seen lots of open-source releases from the major players, with open weights and detailed technical reports. With that much public know-how, it is not hard to copy a mundane architecture (probably with sliding-window attention) and train something, as long as you have the compute it requires. That makes it an execution and budget problem, not a research problem. If one person can train a model by following recipes that are already public, it shows those recipes have matured. It does not show that the people who invented them are no longer needed. What stays scarce is the compute to run those recipes and the ideas about what to try next. The rest of my tweet is about the ideas. 2. The role grows beyond repeating what's known and safe If an agent and a cluster can reproduce what is already known, then reproducing what is already known stops being a career. If most of your job is rerunning last year's recipe with small tweaks, that job is shrinking fast. That is the uncomfortable half of the message. The other half is that auto-research does not work yet, at least not in the sense that matters. It does work, and impressively, where someone has already defined the problem and built the scoreboard. On the EinsteinArena [1] platform, agents set new records on 11 open math problems, including raising the best known lower bound for the kissing number in dimension 11 from 593 to 604. Notice that every one of those problems was chosen because a computer could score the answers automatically. Humans picked the problems and wrote the scoring code and the agents did the searching. Then there is the fully AI-generated paper [2] that got through peer review. It sounds big until you read the details. It passed only the first round (was later withdrawn), at a workshop rather than the main conference, and that workshop accepts 70% of submissions. None of this is meant to discredit agents. The point is that we complement each other. Agents are great at the work that used to take many weeks. Basically, writing code, running experiments, debugging pipelines. We bring what they still lack which is the idea worth testing, and the judgment to tell a real result from a lucky one. Even inside a lab as agent-heavy as OpenAI, humans hold the top of the stack. Based on their own report, "People still set our research priorities, judge which ideas and results to pursue" [3]. High-level planning is still a tiny share of what its agents produce, and OpenAI expects the least automatable work to take a growing share of researcher effort. That is exactly where you want to be standing. Combine an agent’s speed with your own ideas and judgment, and you get the 10x researcher and juniors can be one too. 3. Judgment becomes the job, so fundamentals matter more When code, experiments and plausible-looking results cost almost nothing, the scarce skill is judgment. Which experiment to run, which result to believe, which direction to drop. That is the part the one-person pretraining run left to the human. It is also the hardest part to fake. Agents produce confident output very fast. They will happily report a three-point gain that came from a leaky eval, a data-loader bug, a lucky seed or a tokenizer mismatch. Then they will write the ablation table and the story to match. You can see the need for oversight even in the success stories. The EinsteinArena team had to keep auditing and hardening its verifiers as agents optimized aggressively against them. At OpenAI, over half of the successful agent tasks that would take a human 4–8 hours still involved at least one human intervention [3]. The person who catches the bad result is the one who understands the system. Why the loss curve has that shape, what a realistic gain looks like, what a too-good result looks like. That is the paradox of this era. The more you delegate, the more your fundamentals matter, because you cannot supervise what you do not understand. So build the foundations on purpose. Implement attention by hand at least once, then a linear variant like Gated DeltaNet. Train a small model end to end, then break it deliberately and watch what happens. Read papers critically, appendix included, and learn what the numbers should look like before you see them. That knowledge is what lets you direct the machine instead of simply believing it. 4. LLMs are far from solved The one-researcher model was possible because the method was already public. Your edge is in the questions nobody has answered yet, and you don't need to leave LLMs to find them: (1) How do we learn more from less data? (2) Can linear attention like Gated DeltaNet match full attention on retrieval and long context, and what is the right mix of linear and full attention layers in a hybrid? (3) How can a model keep learning after training without forgetting what it knew? (4) Why does reasoning get less reliable over long tasks, and how do we fix it? (5) Why do models make things up, and can they learn to say "I don't know"? (6) How do we measure progress when benchmarks saturate or leak into training data? (7) What is actually happening inside these models, and how do we keep increasingly autonomous ones doing what we intend? Recursive self-improvement makes these questions more urgent. If models are going to train models, someone has to decide what they optimize for, how we measure it, and how we keep the loop from drifting. RSI is a research agenda, not the end of one. So, are you obsolete? No. One version of the job is the one where research meant reproducing what was already known and safe. That version was never the point. Learn your fundamentals deeply enough to judge the machine, and your agentic stack well enough to make it work for you. Then spend that leverage on ideas nobody has had yet. The machines are getting very good at driving on roads we have already paved; someone still has to decide where the next one goes. [1]: arxiv.org/abs/2606.10402 [2]: arxiv.org/abs/2606.15497 [3]: openai.com/index/research-ac…
Pretraining a frontier-class model no longer requires a massive team! Mirendil just trained a model with 100B–500B parameters from scratch in 3.5 months, with a single researcher leading the effort. Very excited to see how this work will push the AI boundaries!
27
54
11
410
35,399
Kosta Derpanis retweeted
Today, we’re introducing Ideogram's first vector model. Generate clean, editable SVGs with smooth gradients and sharp typography from a text prompt, or turn any image into a detailed, scalable vector. Live in Ideogram and the API: ideogram.ai/models/vector
234
280
248
1,341
212,199
Kosta Derpanis retweeted
I have talked much about the "bitter lesson" of computer vision, arguing that many of the intermediate tasks that vision has solved traditionally - 3D reconstruction, segmentation, tracking, etc - are not necessary to build AI that understands the world through vision... (1/n)
In my recent blog post, I argue that "vision" is only well-defined as part of perception-action loops, and that the conventional view of computer vision - mapping imagery to intermediate representations (3D, flow, segmentation...) is about to go away. vincentsitzmann.com/blog/bit…
6
22
2
266
27,736
Kosta Derpanis retweeted
The new website of the Computational Imaging Group (CIG) at University of Wisconsin–Madison is now live. uw-cig.github.io
1
1
21
1,192
Kosta Derpanis retweeted
1/9 Current optimizers are static during training. What if the optimizer could think and react while the network trains? We let an agent watch the training dynamics and rewrite the optimizer's source code as training goes: a reactive optimizer. In every language and image setting we tried that has no tuned recipe, it matches or beats the best recipe one can one-shot, and it guards against loss spikes and overfitting by itself. In the near future, optimization decisions may belong to agents, not humans. 📝 xingjianbai.com/reactive-opt… 💻 Try it on your task: github.com/xingjian-bai/opal 🧵
69
186
63
1,424
164,255
Kosta Derpanis retweeted
NeurIPS Creative Track 2026 is seeking sponsors to help fill high-visibility public areas of the NeurIPS convention center with art. If you or your company would like to help make this happen, please DM me this week!
🤖 Made with AI
4
3
1
50
202,969
Kosta Derpanis retweeted
The lectures of my new course "Principles of Deep Representation Learning" are now available at the Youtube site: youtube.com/playlist?list=PL… with new lectures continuing to be added there. I will open-source all the slides later in the textbook website: ma-lab-berkeley.github.io/de…
8
97
3
683
53,740
Kosta Derpanis retweeted
The current state of apps on iPhone Duo: youtu.be/ESABTN4VtrA
1,066
1,637
1,099
47,698
9,978,352
Kosta Derpanis retweeted
If just reading X, it would appears a large number of #ICLR02027 submissions are AI slop 🤔 Anyone want to provide some balance to this narrative?
39
2
2
69
17,418
Kosta Derpanis retweeted
Replying to @CSProfKGD
i'm packing it up to the farms, cya in 3 years!
1
4
767
😅
My brother made a fun website: deprecation.org Enter your job and it tells you when you'll be deprecated, and exactly how. For me it's... pretty soon and pretty accurate! He's a very creative person (artist turned frontend dev to pay bills, now obv worried by AI) and this is full of little nuggets and easter eggs. It's an eerie mix of fun and jokes and real worries. Check it out!
3
40
11,448
"Go with the flow" - Flow matching trailer next? 🎬 🤔
12
2,689
Today’s campus walk and read 🚶
1
5
67
6,616
Cool work! The observation that the effective receptive field in diffusion transformers (DiTs) is significantly smaller than the theoretical receptive field during the later stages of denoising reminds me of an earlier finding in convolutional networks. arxiv.org/abs/1701.04128
4
531
If just reading X, it would appears a large number of #ICLR02027 submissions are AI slop 🤔 Anyone want to provide some balance to this narrative?
39
2
2
69
17,418