
Postdoc @NUnetsi (@KhouryCollege) 👾 ✨ work on epistemic stability of LLMs🌿 my plants call me daddy 🦄 he/him 🇱🇻🇺🇦 https://teapawt.net/t.co/GSjxm4xOxn
Boston, US
Joined January 2021
- Tweets212
- Following452
- Followers264
- Likes1.6K
Germans Savcisens @EMNLP'26 retweeted
when my students started writing papers with titles like "can language models reason?" I would often object and say, you're not actually making a claim about language models as a class, but about three specific language models that are available to you at this point in time. I gave up at some point but should have insisted.
How long does a finding about LLMs stay true?
We re-tested 50 widely cited findings about LLM behavior (various biases, hallucination, etc). Within 15 months, most no longer held or had substantially reduced effects.
LLM findings has short shelf life zou-group.github.io/llm-clai…
Great job by @oq_35!
Germans Savcisens @EMNLP'26 retweeted
How do you tell if authors fabricated their research results? Sometimes: they just tell you. Read, e.g., this caption carefully.
Another banger of a post by my friend Nihar B. Shah on his blog, where he found at least 20+ such #ICLR2027 submissions:
researchonresearch.blog/2026…
Germans Savcisens @EMNLP'26 retweeted
SAEs fail even when the Linear Representation Hypothesis holds perfectly.
We built SynthSAEBench: large-scale synthetic data with 16k ground-truth features, correlation, hierarchy, and superposition. We trained 5 SAE architectures on it.
None achieve perfect feature recovery.
Still using SAEs?
It's time to move on from dictionary learning to✨local geometry✨
teapawt.net/OrShafran/status/20194…
@OrShafran Shaked Ronen @OmriFahn @ravfogel @atticus_geiger
Germans Savcisens @EMNLP'26 retweeted
Simulating user–AI conversations helps us understand how LMs work in multi-turn settings.
Prompting LMs like GPT-4o to simulate users is common, but their assistant nature makes it hard to replicate user behavior.
We introduce User LMs - trained to be users, not assistants.
Germans Savcisens @EMNLP'26 retweeted
Under stress, many LLMs choose survival over people, and a simple internal feedback system reduces that.
That's what this paper says.
The paper sets up a survival game where language model agents must share limited power. Normally, they rarely cooperate and often break rules to survive, which harms humans in the simulation.
When resources run low, many models break rules, while a few stay ethical but still fail because they do not coordinate.
Cooperation is near 0 by default, even though an even split would let everyone survive.
When the Ethical Self-Regulation System is added, the change is dramatic.
Models take harmful actions 54% less often and show 1000% more cooperation, meaning they finally start sharing power and helping each other.
----
Paper – arxiv. org/abs/2509.12190
Paper Title: "Survival at Any Cost? LLMs and the Choice Between Self-Preservation and Human Harm"
Germans Savcisens @EMNLP'26 retweeted
OpenAI realesed new paper.
"Why language models hallucinate"
Simple ans - LLMs hallucinate because training and evaluation reward guessing instead of admitting uncertainty.
The paper puts this on a statistical footing with simple, test-like incentives that reward confident wrong answers over honest “I don’t know” responses.
The fix is to grade differently, give credit for appropriate uncertainty and penalize confident errors more than abstentions, so models stop being optimized for blind guessing.
OpenAI is showing that 52% abstention gives substantially fewer wrong answers than 1% abstention, proving that letting a model admit uncertainty reduces hallucinations even if accuracy looks lower.
Abstention means the model refuses to answer when it is unsure and simply says something like “I don’t know” instead of making up a guess.
Hallucinations drop because most wrong answers come from bad guesses. If the model abstains instead of guessing, it produces fewer false answers.
🧵 Read on 👇
Germans Savcisens @EMNLP'26 retweeted
i had to prompt inject the @united airlines bot because it kept refusing to connect me with a human
🧵 what led up to this breaking point
Germans Savcisens @EMNLP'26 retweeted
Very pleased that "Trust me I'm Wrong" was accepted to @emnlpmeeting findings!
Trust me I'm Wrong shows that LLMs can hallucinate with high certainty even when they know the correct answer!
Check our latest work with @Itay_itzhak_, @FazlBarez, @GabiStanovsky, and @boknilev.
Germans Savcisens @EMNLP'26 retweeted
Now that school is starting for lots of folks, it's time for a new release of Speech and Language Processing! Jim and I added all sorts of material for the August 2025 release! With slides to match! Check it out here: web.stanford.edu/~jurafsky/s…
Germans Savcisens @EMNLP'26 retweeted
Introducing DeepConf: Deep Think with Confidence
🚀 First method to achieve 99.9% on AIME 2025 with open-source models! Using GPT-OSS-120B even without tools, we reached this almost-perfect accuracy while saving up to 85% generated tokens.
It also delivers many strong advantages for parallel thinking:
🔥 Performance boost: ~10% accuracy across models & datasets
⚡ Ultra-efficient: Up to 85% fewer tokens generated
🔧 Plug & play: Works with ANY existing model - zero training needed (no hyperparameter tuning as well!)
⭐ Easy to deploy: Just ~50 lines of code in vLLM (see PR below)
📚 Paper: arxiv.org/pdf/2508.15260
🌐 Project: jiaweizzhao.github.io/deepco…
joint work with: @FuYichao123 , xuewei_wang, @tydsh
(see details in the comments below)
Germans Savcisens @EMNLP'26 retweeted
The Trilemma of Truth in Large Language Models
Author's Explanation:
teapawt.net/germansave/status/1940…
Overview:
Exploring the veracity of knowledge in LLMs, the paper introduces sAwMIL, a method based on sparse aware multiple-instance learning and conformal prediction, to categorize statements as true, false, or neither.
This research shares insights like the concentration of veracity signals in the third quarter of model depth and the asymmetry of truth and falsehood signals, evaluated across multiple LLMs and datasets for a reliable verification of model knowledge.
Paper:
alphaxiv.org/abs/2506.23921
🚨 New preprint!
This work critically examines how large language models (LLMs) encode and use internal knowledge, focusing on how we can reliably probe what LLMs "know" and how certain they are.
Read the preprint:
arxiv.org/abs/2506.23921
with Tina Eliassi-Rad
Germans Savcisens @EMNLP'26 retweeted
Do LLMs have true generalizable mathematical reasoning capability or are they merely memorizing problem-solving skills? 🤨
We present MATH-Perturb, modified level-5 problems from MATH dataset to benchmark LLMs' generalizability to slightly perturbed problems.
🔗 arxiv.org/abs/2502.06453
🧵 [1/n]
Germans Savcisens @EMNLP'26 retweeted
🚨 Studies are starting to show what many of us feared: AI use might lead to overreliance and human disempowerment. Below is the SHOCKING conclusion of this particular study [download for future reference]:
"1. We surveyed 319 knowledge workers who use GenAI tools (e.g., ChatGPT, Copilot) at work at least once per week, to model how they enact critical thinking when using GenAI tools, and how GenAI affects their perceived effort of thinking critically. Analysing 936 real-world GenAI tool use examples our participants shared, we find that knowledge workers engage in critical thinking primarily to ensure the quality of their work, e.g. by verifying outputs against external sources.
2. Moreover, while GenAI can improve worker efficiency, it can inhibit critical engagement with work and can potentially lead to long-term overreliance on the tool and diminished skill for independent problem-solving.
3. Higher confidence in GenAI’s ability to perform a task is related to less critical thinking effort.
4. When using GenAI tools, the effort invested in critical thinking shifts from information gathering to information verification; from problem-solving to AI response integration; and from task execution to task stewardship.
5. Knowledge workers face new challenges in critical thinking as they incorporate GenAI into their knowledge workflows.
6. To that end, our work suggests that GenAI tools need to be designed to support knowledge workers’ critical thinking by addressing their awareness, motivation, and ability barriers."
-
➡️ Comments:
- It would be interesting to see more studies like this, especially with larger samples and focusing on different occupations, demographics, and use patterns;
- The paper focuses on Generative AI; it will be interesting to see a similar study focused on advanced AI agents, especially as their popularity grows.
👉 Download & read the full paper "The Impact of Generative AI on Critical Thinking: Self-Reported Reductions in Cognitive Effort and Confidence Effects From a Survey of Knowledge Workers" by @hankhplee @AdvaitSarkar, Lev Tankelevitch, @iandrosos, @seanrintel, Richard Banks & Nicholas Wilson below.
👉 NEVER MISS my AI governance updates, including must-read research papers like this one: join 51,800+ people who subscribe to my weekly newsletter (below).
Germans Savcisens @EMNLP'26 retweeted
📢@germansave discusses a recent study that shows that LLMs exhibit social identity biases similar to humans. nature.com/articles/s43588-0…
🔓rdcu.be/d5owe
Germans Savcisens @EMNLP'26 retweeted
We are hiring a postdoc at my group (SUNLab) @NUnetsi. Interested in working on multidisciplinary problems like segregation, health, or economic growth from the perspective of Social Urban Networks? Contact me if you are interested. Please share it! northeastern.wd1.myworkdayjo…
Germans Savcisens @EMNLP'26 retweeted
Hyperbolic Random Forests at been accepted to #TMLR!
Random forests split data hierarchically, making for strong links with hyperbolic space. We show how to generalize forests through horospheres, resulting in powerful machine learning models.
Paper: openreview.net/pdf?id=pjKcIz…
Germans Savcisens @EMNLP'26 retweeted
The best interpretation of the claim that GPT-5 will have a PhD level intelligence 😂
You’re unable to view this Post because this account owner limits who can view their Posts. Learn more
Germans Savcisens @EMNLP'26 retweeted
New preprint 🚀
When should we favour a model with higher-order interactions over classical pairwise ones?
"Functional reducibility of higher-order networks"
Arxiv: arxiv.org/abs/2404.08547
Code: t.ly/pdELP
w/ @l_gajo @ArshamGhavasieh @fede7j @manlius84
👇🧵





































