what do we do, now that we are happy? audio guy @withdavidai | ex vision guy @tesla
Joined April 2020
- Tweets657
- Following740
- Followers393
- Likes3.1K
Arth retweeted
Everyone thinks creators blow up overnight because you discover them at 10k and soon they’re at 100k
Definitionally very few see the 0 to 10k grind
Just learned about cash basis accounting and am struggling to understand why we've collectively decided to entertain any other definition of revenue.
Every neocloud should be watching this.
What happens when the useful compute you can squeeze out of a GPU improves faster than the hardware itself?
Billions in GPU-backed financing depend on assumptions about future rental rates and residual values. Software breakthroughs could make those assumptions obsolete overnight.
Posting guitar covers has always been a micro mental workout because on one hand I absolutely love playing the guitar but on the other am afraid of people watching me play the guitar.
Arth retweeted
What happened in Robotics TODAY:
- @nvidia GEAR released Flora, a suite of open models to parse the physical world and turn raw footage into rich training signals (1/125 cost of Astra)
- @EkaRobotics raised $150M Series A from @a16z and @FPrimeCapital focused on building vision-force-action (VFA) models
- @microagi released 2 benchmarks that shows evaluates next-action prediction and annotation across a long horizon
- @roboticscorp (Tuesday Labs) introduced Voxel, a droid butler (v cute!)
- Dreamscale released Deployments their inference engine that allows you to serve your own fine tuned ckpts on your robots in real time
Stoked to finally share this with the world!
Most speech benchmarks test clean, scripted audio. Real conversations are messy, overlapping, spontaneous, long-form, and multilingual.
As researchers, we built the benchmark we wished we had.
Introducing DAI-ASR-I18N.
Language had an encoding tax before it had a token tax.
It's common knowledge that UTF-8 uses more bytes for many scripts, but some pack more information into each grapheme depending on the writing family the language comes from.
Does that tradeoff carry into LLM tokenizers?
I explored it here: arthvid.com/field-notes/toke…
Learning linguistics is fun because it highlights just how many tradeoffs that we see in modern text tokenization already appeared in traditional writing systems.
Eg. Chinese, being a logographic language, has a much larger vocabulary size compared to an alphabet based language like English. However since each character is packed with more meaning, sequences are often shorter.
There is another category of languages called abjads, which do something more interesting and seemingly break this tradeoff.
1/7
This suggests an interesting way to think about LLM efficiency.
What if a tokenizer learned not only how to represent text, but what it could safely leave out?
A model might receive a compressed sketch like “cat sat mat” and reconstruct “the cat sat on the mat” using its existing priors.
6/7