abhi retweeted
Next Concept Modeling from @stanford and @togethercompute that performs better than DLMs at similar parameter scale and 21x faster inference.
Text generation now has a third option.
Beyond autoregressive and diffusion models.
@Stanford and @togethercompute proposed Next-Concept Modeling (NCM) that generates text through a hierarchy of compressed codes, building from a coarse representation to finer detail.
These codes are "concepts" in the name - learned representations, not explicit ideas or human-readable outlines.
▪️ NCM works in 2 parts:
1. It starts with a compressed representation.
A multiscale residual VQ-VAE (Vector Quantized Variational Autoencoder) learns to turn text into several levels of discrete codes and reconstruct text from them.
Level 1 is brief and coarse, and higher levels add what the reconstruction still lacks.
2. Then it adds finer detail.
A Transformer generates those codes from coarse to fine, one level at a time, predicting all codes in each level in parallel.
An important note: NCM adds detail at successive scales, doing less repeated work than diffusion models and skipping teacher–student distillation.
In tests, NCM showed:
- 10–31× faster generation than the strongest diffusion baselines, with comparable or better quality.
- Higher Sudoku accuracy than the tested autoregressive and diffusion, using fewer model evaluations.
~4–5× fewer training tokens than strong diffusion baselines.
That's for small models and specific benchmarks. Compares at large language-model scale are not established yet.
Text gen needs this breath of fresh air, and all new ideas are valuable. The NCM approach makes it cheaper, with fewer steps and less work are needed.
abhi retweeted
Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry.
Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come.
But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility.
This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems.
Together, we are building the foundation of the AI economy.
Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. nvda.ws/4hcoq7m
abhi retweeted
Our serverless APIs for open weights models have become exquisite machinery (that we plan to write more about soon!)
One effect is that @togethercompute now one of the largest originators of open tokens in the US, with a wild week over week growth curve that reflects the pace of OSS adoption. Will serve 1.8T tokens today on @OpenRouter alone.
You can get started with $5 — api.together.ai
openrouter.ai/provider/toget…
abhi retweeted
A primer from @togethercompute on how to bring open source models to work alongside or substitute closed models.
abhi retweeted
The Chinese do not need to distill anymore. They've climbed up the proverbially tech tree that to climb higher, they need to do actual frontier work. Distillation is marginal.
Any lobbying or legislation that calls for banning open weight models in the name of "distillation" is total BS. This is a fight against future American innovation.
The labs know that open weight models completely commoditize their revenue generator. They know that we will optimize it to 1/100th the costs. They know that we will stop concentrating our workloads on them. If we start using open models, they lose the understanding of workloads that they can build products around to *replace* your products as they move up to the app layer.
Remember: *they* trained on humanity's data and didn't pay a cent. If you own intelligence, you can tax humanity.
Do not let them fool you.
abhi retweeted
We analyzed Kimi K3 vs. Claude Fable 5 for software engineering tasks using DeepSWE.
Kimi K3 gets you the same performance as Fable 5 at ~35% of the price, and it actually pulls ahead at higher pass@k's. More insights in the thread!
abhi retweeted
We @togethercompute believe intelligence should be abundant, not expensive.
Today we announced our Series C funding of $800m @ $8.3B valuation, to continue to build the world's most efficient platform for generative AI.
Thanks @nikogallogly for telling our story in @nytimes!
shorturl.at/SooOP
abhi retweeted
71% of Trump voters and 87% of Harris voters support admitting more high skilled immigrants. The “anti-high skilled immigration” is just losers amplified by bots on Twitter
abhi retweeted
Anyone – of any race, creed or nationality – who came to America and worked like hell to contribute to this country will forever have my respect.
America is the land of freedom and opportunity. Fight with every fiber of your being to keep it that way!
🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸🇺🇸
abhi retweeted
The reason I’m in America along with so many critical people who built SpaceX, Tesla and hundreds of other companies that made America strong is because of H1B.
Take a big step back and FUCK YOURSELF in the face. I will go to war on this issue the likes of which you cannot possibly comprehend.
abhi retweeted
High skilled immigration has been central to America leading the world in tech.
The biggest misunderstand about high skill immigration stems from people thinking that the market opportunities in tech, and tech-adjacent fields, are zero sum. This essentially imagines innovation is finite and we’re all fighting over the same job or opportunity pool.
This may be true of a few very legacy, slow growth industries, but it’s categorically not true for any important industry in the past 50 years or the next 100. Biotech, AI, advanced manufacturing, software, EVs, new energy sources, and dozens of other fields of the future are our high growth industries. And there’s no inherently fixed volume of companies or talent that the market needs.
Tesla being started or not started in America is the difference between 100,000’s of jobs here - and leading in EVs globally - and not. Apple being started here is the difference between potentially millions of jobs being here - and leading consumer electronics globally - and not. You could go through this list all day long.
Tech is not zero sum. More startups, pursuing more ideas, ultimately create more innovation and ultimately more jobs and prosperity. And that means you need the right talent to both work at these companies, and start the next ones. High skilled immigration has directly made America dominant technically and thus economically, and create far more jobs in America for others than are supposedly displaced.
Even briefly imagining the alternative scenario, it’s obvious how disastrous this would be. The demand from tech companies for this top talent will remain, yet America won’t benefit directly from their hiring. That talent will go to another company that competes with the US and makes our dominance harder to maintain. You’re just increasing the odds you have more competition in the future.
And even in the “best case” scenario (for our competitiveness) where a larger company like Google hires the same people internationally that would have otherwise moved here, when that person leaves Google to start their next company, it will be in their country of origin, not America.
This is how you lose the tech war within one or two generations. There’s simply no good game theory in anything that reduces our talent access.
Yes, we absolutely have and need to continue to educate and train incredible talent that grows up in the US, but equally having access to the world’s smartest talent has always been a huge advantage for America.
abhi retweeted
I don't think I've ever seen a Twitter dispute that was so one-sided. All the people who understand tech, on both right and left, are on one side, and the other side is just anonymous accounts and political commentators.
abhi retweeted
Immigrants are responsible for many of our most important technologies and companies.
IMO, attracting the world’s best talent to build here is one of America’s greatest strengths.
I don’t believe you can truly be pro-America if you’re anti-skilled immigration.
abhi retweeted
One of the best things the U.S. can do is make high-skill immigration easier. @levie is right.
It is awful that the wait time for a green card can be over a decade, and that after waiting years someone can still be forced to leave simply because they lost a job. Fixing this is both an economic and a moral issue.
A rigorous economic analysis (by Pierre Azoulay and collaborators) shows that immigrants create more jobs than they take. So to create jobs for Americans, lets let more immigrants in!
High skilled immigration has been central to America leading the world in tech.
The biggest misunderstand about high skill immigration stems from people thinking that the market opportunities in tech, and tech-adjacent fields, are zero sum. This essentially imagines innovation is finite and we’re all fighting over the same job or opportunity pool.
This may be true of a few very legacy, slow growth industries, but it’s categorically not true for any important industry in the past 50 years or the next 100. Biotech, AI, advanced manufacturing, software, EVs, new energy sources, and dozens of other fields of the future are our high growth industries. And there’s no inherently fixed volume of companies or talent that the market needs.
Tesla being started or not started in America is the difference between 100,000’s of jobs here - and leading in EVs globally - and not. Apple being started here is the difference between potentially millions of jobs being here - and leading consumer electronics globally - and not. You could go through this list all day long.
Tech is not zero sum. More startups, pursuing more ideas, ultimately create more innovation and ultimately more jobs and prosperity. And that means you need the right talent to both work at these companies, and start the next ones. High skilled immigration has directly made America dominant technically and thus economically, and create far more jobs in America for others than are supposedly displaced.
Even briefly imagining the alternative scenario, it’s obvious how disastrous this would be. The demand from tech companies for this top talent will remain, yet America won’t benefit directly from their hiring. That talent will go to another company that competes with the US and makes our dominance harder to maintain. You’re just increasing the odds you have more competition in the future.
And even in the “best case” scenario (for our competitiveness) where a larger company like Google hires the same people internationally that would have otherwise moved here, when that person leaves Google to start their next company, it will be in their country of origin, not America.
This is how you lose the tech war within one or two generations. There’s simply no good game theory in anything that reduces our talent access.
Yes, we absolutely have and need to continue to educate and train incredible talent that grows up in the US, but equally having access to the world’s smartest talent has always been a huge advantage for America.
abhi retweeted
Number of illegal aliens/year: 3+ million.
Number of employment visas/year: 140k.
As a start, maybe we should focus on the 95% where we all agree.
abhi retweeted
Can we staple a US green card to every degree earned at an American university? Instead of sending the best and brightest top talented graduates away after they’ve been admitted to our top schools and graduated with a world-class education, let’s keep them in the USA to fuel our economy and strengthen our society. ❤️🇺🇸
