Postdoc at @ETH_en. Formerly, PhD student at @Cambridge_Uni.
Brasília, Brazil
Joined November 2009
- Tweets1.2K
- Following327
- Followers1.9K
- Likes2.4K
Pinned Tweet
checkout our new paper about the superficial alignment hypothesis :) we use algorithmic information theory to formalise this hypothesis, we unify prior work on the topic, and show how post-training affects it!
follow @tvergarabrowne for more great work like this in the future!
Check out our new #EMNLP2026 paper! Our method lets your model use language-specific tokenisers while still sharing a vocabulary across languages!
@suchirsalhan will present it today at EMNLP :)
Our new paper, LangMAP: A Language-Adaptive Approach to Tokenization, is out at #EMNLP2026 in Budapest! We introduce a way to adapt tokenization to individual languages without changing a pretrained model’s vocabulary. 🧵
Tiago Pimentel retweeted
🚨 New paper!
What if LLMs could reason like we do when solving Sudoku: keep a few candidates per cell and iteratively narrow them down?
Introducing Binary Discrete Diffusion for language and reasoning tasks: it denoises sets of candidate tokens using binary representations! 🧵
Happening soon :) tokenization-workshop.github…
Announcing our first invited speaker at TokShop!
We're thrilled to welcome Tiago Pimentel @tpimentelms (ETH Zürich) for a talk on:
"How much does tokenisation impact language models?"
🧵
Tiago Pimentel retweeted
Check out the paper on @askalphaxiv !
alphaxiv.org/abs/2609.tokeni…
If you've been meaning to dive into tokenisation, we have a beefy new survey for you :) Led by @marco_computers and put together by an amazing team. Check it out 👇
Tiago Pimentel retweeted
Please support our call asking frontier LLM providers to share what they’re doing for AI safety, what works and what doesn’t. Many sharp people want to help close the capability–safety gap. Transparency will help them see where their work is needed most.
make-safety-open.github.io
Tiago Pimentel retweeted
🎙️ Excited to announce our second invited speaker: Amir Zamir @zamir_ar, Assistant Professor at EPFL!
Talk: "(Multimodal) Flexible-Length Tokenization"
Looking forward to a talk that pushes tokenization beyond text 👀
Why is BPE so good: the compression objective or the step-by-step merging? 🤔 Standard tokeniser comparisons (eg, BPE ⚔️ UnigramLM) conflate the two. Our #emnlp2026 paper separates tokenisation objective from search procedure and shows BPE's merging strategy is what matters 🧵
ALT Paper title, diagram with 2x2 diagram with tokenisation objectives and search procedure, and a plot showing that bottom-up methods achieve better bits-per-byte than top-down.
When measuring grammaticality performance (in BLiMP), though, no design choice performs consistently better. Better compression thus does not seem to buy you better grammaticality.
We hope future work in tokenisation will ask two questions: what should we optimise, and how should we search? Our results suggest the second also deserves some love!
Paper: arxiv.org/abs/2609.19145
Code: github.com/Ahmetcanyvz/comp-…
First rule of academic writing: don't assume readers can think. Spell everything out #iclr2027
Tiago Pimentel retweeted
Why does the induction head emerge abruptly? It's because of the geometry of the loss landscape: an almost flat plateau around initialization, followed by a sharp drop.
We study this in more detail in this paper you might find interesting:
arxiv.org/abs/2511.01033
Tiago Pimentel retweeted
Announcing our first invited speaker at TokShop!
We're thrilled to welcome Tiago Pimentel @tpimentelms (ETH Zürich) for a talk on:
"How much does tokenisation impact language models?"
🧵
Tiago Pimentel retweeted
🐝New preprint🐝
If you want open source speech technology that's actually optimized for your target language, BuzzASR might be your best option! All models on Huggingface 🤗
Congrats to @realshivamsingh on an amazing first paper!!
Appearing in EMNLP Findings.
Tiago Pimentel retweeted
This kind of ongoing accountability is going to be open up a lot of really valuable possibilities for safety, and I'd love to see similar things elsewhere:
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must…
Definitely go work with Edoardo if you're searching for a position :)
I am looking for 1 postdoc and 2 fully funded PhDs at Imperial @ICComputing to join my @ERC_Research project AToM ⚛
We aim to build models that learn to compress what they perceive and remember, for permanent memory, longer horizons, and >10x speedups
ducdauge.github.io/atom/
Tiago Pimentel retweeted
You can train your own Parity-aware BPE tokenizer in the HuggingFace library! See the docs for an example :) huggingface.co/docs/tokenize…
Congrats to @clara__meister for getting the Parity-Aware BPE tokenizer merged into Hugging Face tokenizers!
Learn more about it here: alphaxiv.org/abs/2508.04796
Use it like this:
from tokenizers.pre_tokenizers import ByteLevel
from tokenizers.trainers import ParityBpeTrainer
tokenizer = Tokenizer(BPE())
tokenizer.pre_tokenizer = ByteLevel()
# Plain text files: one iterator per language
def lines(path):
with open(path) as f:
yield from f
trainer = ParityBpeTrainer(num_merges=32000, variant="base")
trainer.train_from_iterator(
tokenizer,
train_iterators=[lines("en.txt"), lines("de.txt"), lines("zh.txt")],
dev_iterators=[lines("en_dev.txt"), lines("de_dev.txt"), lines("zh_dev.txt")],
)
tokenizer.save("parity_bpe_tokenizer.json")
Tiago Pimentel retweeted
I am looking for 1 postdoc and 2 fully funded PhDs at Imperial @ICComputing to join my @ERC_Research project AToM ⚛
We aim to build models that learn to compress what they perceive and remember, for permanent memory, longer horizons, and >10x speedups
ducdauge.github.io/atom/