@BdsLoick

FAT5 (Flash Attention T5) boy @huggingface Fellow 🤗

 France
huggingface.co/lbourdois
Joined October 2021
  • Tweets369
  • Following255
  • Followers305
73Media
New blog post on @huggingface! An introdution to Trimming ✂️ a little-known but highly effective model reduction method. We achieved up to 87.24% size reduction while preserving performance 🧵
1
4
10
663
Loïck BOURDOIS retweeted
Paulo Moura@paulomourajOct 19
We surfed the dense and late-interaction waves… now it's time to SlideOn sparse retrieval 🐕 🚀 Meet SlideOn and mSlideOn, fully open SOTA sparse retrievers at 174M and 339M parameters, trained by @LightOnIO R&D Models: huggingface.co/collections/l… Blog: huggingface.co/blog/lightona…
7
25
7
74
7,234
Loïck BOURDOIS retweeted
Antoine Chaffin
@antoine_chaffinOct 12
The future is sparse (and open releases call for open release, very happy both our models and our data is useful to create other very cool open models!)
Loïck BOURDOIS@BdsLoickOct 12
New open-source SOTA for French sparse retrieval! 👀 First, for those unfamiliar with this type of models, a sparse embedding model offers the advantage of being able to retrieve synonyms for words in a user query, whereas good old BM25 only retrieves exact matches.
1
1
27
1,295
Loïck BOURDOIS retweeted
tomaarsen@tomaarsenOct 12
I always love models trained for one non-English models, and it's no exception here. @BdsLoick from @AI_AlphaEdge trained a French Sparse Embedding model for retrieval. It's SOTA at its size (150M), very impressive as sparse embedding models are notoriously hard to train. ->🧵
Loïck BOURDOIS@BdsLoickOct 12
New open-source SOTA for French sparse retrieval! 👀 First, for those unfamiliar with this type of models, a sparse embedding model offers the advantage of being able to retrieve synonyms for words in a user query, whereas good old BM25 only retrieves exact matches.
3
2
24
1,590
New open-source SOTA for French sparse retrieval! 👀 First, for those unfamiliar with this type of models, a sparse embedding model offers the advantage of being able to retrieve synonyms for words in a user query, whereas good old BM25 only retrieves exact matches.
5
7
3
30
3,819
more replies
What we’re sharing: We’ve written a short blogpost sharing our experience with this successful replication. We are also making the benchmark and the trained model available on our HF organization 🤗 Blogpost: huggingface.co/blog/alphaedg… Collection: huggingface.co/collections/a…

Sparseup-fr: A French replication of Linkup's sparse embedding model

A Blog post by AlphaEdgeAI on Hugging Face

huggingface.co
1
5
122
References: Original Sparseup authors: @thibault_formal, Pierre Erbacher, @cosimorulli1 Original backbone and data authors: @antoine_chaffin, @raphaelsrty, @paulomouraj, @AmelieTabatta Sentence transformers GOAT: @tomaarsen GPU providers (my boss): @AI_AlphaEdge, @TheoHubLab
10
115
Loïck BOURDOIS retweeted
Basile Terver
@BasileTerv987Oct 6
Everything is open: code to train, plan and reproduce every result, 93 checkpoints (4 sim envs + DROID) and the datasets. 📄 arxiv.org/abs/2610.06805 🌐 h-jepa.com 💻 github.com/kevinghst/H-JEPA 🤗 huggingface.co/jepa-world-mo…

H-JEPA: End-to-End Learning of Hierarchical World Models for...

Long-horizon planning with latent world models requires reasoning across timescales and levels of abstraction. Existing task-agnostic JEPA world models predict and plan at a single timescale or...

arxiv.org
5
11
630
Loïck BOURDOIS retweeted
Loïck BOURDOIS@BdsLoick20 Mar 2025
We introduce FAT5 (Flash Attention T5) ⚡ An implementation of T5 in PyTorch with UL2 objective optimized for GPGPU for both training and inference thanks to 13 differents optimizations. Unroll to find out more! 🧵👇 [1/8]
1
1
7
1,016
Replying to @BdsLoick @IBM @Microsoft @GoogleDeepMind @baai @qwen @orionweller @vllm @Google @Meta @huggingface @Nils_Reimers @jaseweston
From these 16 families, we generated more than 5,500 monolingual models in 124 different languages.
1
3
112
Big thanks to my HF Fellows bros for multilingual evaluation @tomaarsen, Bram Vanroy, @christopher, @w00jun_ @mrm8488, @prithivMLmods and to @AI_AlphaEdge for the time dedicated to this project 🙏 Links 👇 Blogpost: huggingface.co/blog/lbourdoi… Models: huggingface.co/spaces/alphae…

Introduction to Trimming ✂

A Blog post by Loïck BOURDOIS on Hugging Face

huggingface.co
2
4
413
Load more