PhD student at @HD_NLP. My research interests are graphs 🕸 and NLP 🗣, more specifically graph-based reasoning 🧠 for argumentation 👥
Heidelberg, Germany
Joined September 2022
- Tweets54
- Following136
- Followers130
- Likes186
Pinned Tweet
Graph Language Models: the child of LMs🗣 and GNNs🕸
📜arxiv.org/abs/2401.07105
TLDR we modify Self-Attention such that a LM becomes a graph transformer. The new architecture enables graph reasoning, while the pretrained parameters allow text understanding
To appear at #ACL2024
Moritz Plenz retweeted
How and when do multilingual LMs achieve cross-lingual generalization during pre-training? And why do later, supposedly more advanced checkpoints, lose some language identification abilities in the process? Our #ACL2025 paper investigates.
Debates aren’t always black and white—opposing sides often share common ground. These partial agreements are key for meaningful compromises.
Presenting “Perspectivized Stance Vectors” (PSVs) — an interpretable method to identify nuanced (dis)agreements
📜 arxiv.org/abs/2502.09644
For example, we find signature perspectives that authors tend to agree, disagree, or be orthogonal on. Many more details, considering e.g., different stakeholder groups are in the paper.
Many thanks to @PhilippHeinisch, Janosch Gehring, @pcimiano and @AnetteMFrank (@HD_NLP) for their great contributions.
Please reach out if you have any questions, and see you in Albuquerque at @naacl 2025 🥳
Moritz Plenz retweeted
✨New Paper✨ The Mystery of Compositional Generalization in Graph-based Generative Commonsense Reasoning #EMNLP2024
📜 aclanthology.org/2024.findin…
🖼️ Poster Session F (Riverfront Hall), Nov 14 @ 10:30
This Thursday at 15:00 UTC I will present at @roosh_circle's Papers Club. Many thanks for inviting me!
We cover methods for Text-Attributed Graphs, in particular Graph Language Models.
The talk is targeted towards anyone interested, so everyone is welcome to join 🤗
Link in 🧵
Moritz Plenz retweeted
How to make powerful LLMs understand graphs and their structure?🕸️ With Graph Language Models!
Take a pre-trained LLM and fit it with the ability to process graphs. Watch if you're curious how:👇
📺 youtu.be/JcHeaONGbmQ
(Hint: it's about position embeddings, as @MoritzPlenz explained at #ACL2024 🔴)!
Work from @HD_NLP .
🤗Our models are now on Huggingface and can be run with only a couple of lines of code: huggingface.co/models?other=…
Graph Language Models: the child of LMs🗣 and GNNs🕸
📜arxiv.org/abs/2401.07105
TLDR we modify Self-Attention such that a LM becomes a graph transformer. The new architecture enables graph reasoning, while the pretrained parameters allow text understanding
To appear at #ACL2024
Moritz Plenz retweeted
Our @MoritzPlenz from @HD_NLP in front of numerous visitors of his poster, presenting his new work on Graph Language Models 🕸️🗣️ at #ACL2024NLP. GLMs unify the advantages of LLMs and GNNs on structure-based tasks. Hold on, Moritz, to the end of the session! 😅🤩
8️⃣ (iii) Ablations on WikiData and Wikipedia show that GLMs equally leverage both modalities, successfully fusing the complementary information.
More details can be found in our paper. Our code is available and ready to encode your TaGs: github.com/Heidelberg-NLP/Gr…
9️⃣ Many thanks to @AnetteMFrank for the active contribution and guidance. @HD_NLP
Also, many thanks for reading this far - reach out if you have any questions or if you are attending ACL and want to say hi 🙋