More social alt of @ezyang

Joined April 2022
Low-background YouTube Music is music produced before 2026, the date when AI generated music became good enough to mass proliferate across the catalog.
7
748
However, AFAICT, there was no FP theorems added, so there's no proof that the new implementation actually fixes the FP precision problem that was claimed to be fixed.
1
1
13
1,290
It is probably possible to prove something along these lines! But what exactly is the theorem you want to prove? If you tried to prove absolute error ≤ 10⁻⁷, well, that will admit the buggy implementation! There's a reason your theorem statements are considered "trusted".
1
1
12
1,263
Anyway. I'm super excited about all of the formal verification projects popping up, at PyTorch HQ we're trying to figure out what exactly we want to do in this space, but I've also been doing a lot of FM educating recently. Hope this helps the discourse!
1
21
1,148
(Aside: There actually is one theorem about tensors containing floating point numbers but it's "did we define CE correctly".)
1
1
7
1,502
The fix commit switches over to the stable formulas. The (rightly!) requires adjusting the existing real number proofs, since you're now proving things about a different mathematical formula, but not the statements themselves which remain true (good!)
1
1
8
1,394
It also proved some theorems about logSoftmax, like that the stated JVP is correct and the conventional numerical stability algebraic rewrite is correct. But importantly, these theorems were all proved on tensors handling REAL numbers.
1
1
13
1,967
Real numbers are nice and beautiful and mathematicians love proving theorems with them and it will never tell you anything about the log softmax problem, which is entirely an artifact of floating point representation.
1
2
22
1,754
So the log_softmax bug arxiv.org/abs/2610.31642 that's been making the rounds. TorchLean also had the same problem (fixed in github.com/lean-dojo/TorchLe…). How come? This example is pretty illustrative of the proper mindset you need when working on formal verification. 🧵
6
25
5
230
23,888
LeanTorch implemented ops in Lean. Implementing an algorithm in a theorem proving capable language certainly gives you some guarantees, e.g., it is terminating (if you didn't use "partial"), it is well-typed (this can mean a lot with fancy dependent types). Not bug free!
1
1
19
2,548
A naive understanding of formal verification might be something like, "Well, if I do it in Lean, then it can't have bugs." WDYM "do it"? WDYM "can't have bugs"? This is entirely the question.
2
3
30
2,729
Claude you are so cool
1
32
1,962
Trying out this as an organizing principle for frameworks, still not sure though...
3
21
2,200
It's started, I have begun to be able to recognize AI generated music
2
7
928
Are your training runs APPLICATIVE or MONADIC 🤗
2
11
1,425
a turn of phrase, ruined forever
1
26
2,390
for sale: one accumulator, never reopened
1
10
613
(the paper is probably fine I just hate ai writing lmao)
7
405
arxiv.org/pdf/2609.34272v1 I am annoyed that I have to read this but the abstract (at least) is LLM slop text
4
2
85
40,121
Dipping my toe into the world of reward hacking and it seems like... You could just ask your agent not to reward hack????
10
2
46
4,827