Shumin Deng

🚀 Excited to share SkillX: Automatically Constructing Skill Knowledge Bases for Agents! It automatically converts agent trajectories into reusable, plug-and-play skills — making them transferable across agents and environments. We are also planning to integrate SkillX into the SkillNet series, aiming to build a unified and scalable ecosystem for skill-centric agent intelligence. #LLM #Agents #NLP #AI #Skills #SkillX 📖 Paper: huggingface.co/papers/2604.0… 🔗 Code: github.com/zjunlp/SkillX 🧩 Motivation LLM agents should learn from experience, but today, most self-evolving agents still learn in isolation. They repeatedly rediscover similar behaviors from limited data, leading to: 🔹 redundant exploration 🔹 weak generalization 🔹 capability bottlenecks tied to the base model So the key question is: What form of experience is actually reusable across agents and environments? 💡 Our answer: Skills! But structured hierarchically! We propose SkillX, an automated framework for building a reusable Skill Knowledge Base (SkillKB). Instead of storing raw trajectories, insights, or workflows alone, SkillX organizes experience into 3 levels of skills: 1️⃣ Planning Skills High-level task organization: ordering, decomposition, dependencies 2️⃣ Functional Skills Reusable tool-based subroutines for completing subtasks 3️⃣ Atomic Skills Low-level tool usage patterns, constraints, and failure-prone details This makes agent experience more compact, composable, and transferable. ⚙️ How SkillX works SkillX constructs the skill library through 3 synergistic components: 1. Multi-Level Skills Design 2. Iterative Skills Refinement 3. Exploratory Skills Expansion 🔍 Why is this useful? Unlike long-context skill formats that require complex sandboxing and progressive interaction, SkillX uses a lightweight, itemized representation: ✅ retrieve with a simple retriever ✅ inject once into the system prompt ✅ easier transfer across base models ✅ lower execution burden for weaker agents 📊 Results Using GLM-4.6 to automatically build the skill library, we evaluate transfer on challenging long-horizon interactive benchmarks: ● AppWorld ● BFCL-v3 ● τ2-Bench When plugged into weaker base agents like Qwen3-32B, SkillX brings ~10 point improvements and also improves execution efficiency. ⚡ 🧠 Key takeaway ● Not all “experience” transfers equally well, and the representation matters. ● Hierarchical skills are a powerful abstraction for turning isolated agent experience into reusable knowledge. ● Stronger agents can build the skills, weaker agents can reuse them, and agents no longer need to keep learning everything from scratch. ✨ Additional findings ● Functional skills contribute the most to performance gains ● Planning skills often reduce execution steps ● Atomic skills are crucial for clarifying tool constraints and common failure modes ● Iterative refinement further improves the skill library ● Experience-guided expansion discovers more novel skills than random exploration 📦 We will release the optimized plug-and-play skill library to facilitate future research on reusable agent skills. Feedback, discussions, and collaborations are very welcome! 💬

Catch us at the poster/oral — happy to chat and exchange ideas! #ACL2025

Has the over-forgetting of large language models led to "aphasia"? Our latest work, ReLearn: Unlearning via Learning for Large Language Models, brings a solution! 🚀 Paper: arxiv.org/abs/2502.11190 Code: github.com/zjunlp/unlearn ReLearn adopts a forward learning strategy rather than traditional disruptive reverse optimization, effectively forgetting sensitive information while avoiding excessive suppression and eliminating the issue of generating repetitive and meaningless words. ✅ Key Advantages: 🗣️ Improved Language Fluency: Generates fluent and relevant text, avoiding the "aphasia" phenomenon. 🔒 Balance Knowledge Forgetting and Retention: Effectively forgets sensitive information while retaining general knowledge. 📊 New Evaluation Framework: Introduces KFR, KRR, and Linguistic Score to comprehensively measure forgetting effects and linguistic quality. 🤯 In-depth Mechanistic Analysis: Analyzes the cognitive conflicts caused by reverse optimization from the perspective of knowledge distribution and memory, providing a fresh understanding of the unlearning mechanism in large language models. Dive deeper into ReLearn, making your large language models “forget” smarter and “express” better! 👉#LLMs #Unlearning #ResponsibleAI  #PrivacyProtection #NLP #KnowledgeEditing

Over the past year, #KnowledgeEditing has experienced rapid development. As the new year begins, I’ve taken some time to reflect on the progress of this field and share my thoughts on its future directions. I look forward to discussing and collaborating with everyone to further advance this area. 🛠 Progress in Knowledge Editing: 1. Scenarios: In addition to updating the knowledge of LLMs, many works have begun exploring knowledge editing as a means to control model behavior, promoting safer and more controllable generation while enabling capabilities like unlearning. 2. Side Effects: Many works have started to reflect on the fundamental causes of the side effects of knowledge editing and have explored various methods to mitigate them. Editing LLMs (parameter-altering) can lead to overfitting, where models assign disproportionately high importance to edited content and disrupt attention mechanisms, reducing generalization and general abilities. Whether the model has truly updated its relevant knowledge remains questionable. 3. Practicality: While knowledge editing has expanded to fields like software engineering and multimodal tasks, its real-world impact remains limited. 💡 Key Reflections: 1. The field's foundational goal—knowledge updates—has seen limited success outside areas like AI safety. This raises questions about how to better align methods with practical needs. 2. Mechanism research is lagging. Without clear insights into why knowledge editing works (or doesn’t), efforts to improve models risk being akin to “blind men describing an elephant.” 📈 Future Directions: 1. Evaluation: We need a set of metrics/benchmarks to evaluate whether an edited LLM behaves properly, that is, to achieve a balance between generalization and side effects. 2. Steering: Steering vectors (with SAE) are emerging as a promising approach for interventions in model behaviors, particularly in domains like safety and personality alignment. These methods demonstrate the potential to achieve precise control with minimal impact on overall model performance. Furthermore, they may pave the way for bridging the gap between prompts and model parameter updates, enabling prompt-driven, parameterized behavior adjustments within the model. 3. Agent Memory Updates: The debate between symbolic and parametric memory for AI agents is ongoing. Knowledge editing techniques can offer a unified approach to memory updates, bridging the gap between updating both the model's internal memory and external memory. Memory updates may enhance reasoning capabilities over the long term, fostering the gradual evolution of System 2-like slow thinking processes. 4. Mechanism Interpretation: Deepening our understanding of model mechanisms is essential. Currently, research on the mechanisms of LLMs—such as neurons and circuits—lacks systematic exploration. It also fails to explain phenomena like the dynamic acquisition and forgetting of knowledge, as well as higher-order cognitive behaviors such as slow-thinking reasoning. 5. Interdisciplinary: Drawing inspiration from cognitive/brain science, we may: design the next generation of model architectures and model updating paradigms; potentially simulate human brain behavior based on neural networks to construct an electronic digital twin brain, enabling better solutions (e.g., neuromodulation) to problems in neuroscience and cognitive science. If one day machines truly awaken to self-awareness, understanding their mechanisms and having the means to control them will be a critically important technology. 🎉 Exciting News: We’re thrilled to announce that EasyEdit2 is currently in development! This next-generation toolkit will integrate steering capabilities to enable control over model behavior. Stay tuned for updates, and we welcome the community to explore and contribute: github.com/zjunlp/EasyEdit Let’s continue pushing the boundaries of #KnowledgeEditing, tackling its challenges, and exploring its vast potential to redefine AI adaptability and usability. #LLM #AI #NLP #EasyEdit #LLM #ModelEditing #KnowledgeEditing

We are going to present the following works at #NeurIPS2024 covering knowledge editing, mechanism and agents. Welcome to discuss with the presenters @dsmall2apple1 @yyzTodd ! (ps: I'm not attending) [Knowledge Editing] WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models Location: East Exhibit Hall A-C #3403 Time: Wed 11 Dec 4:30 p.m. PST — 7:30 p.m. PST Also at Foundation Model Interventions Workshop Location: West Meeting Room 121, 122 Time: Sun 15 Dec, 8:15 a.m. PST [Knowledge Mechanism] Knowledge Circuits in Pretrained Transformers Location: East Exhibit Hall A-C #3211 Time: Thu 12 Dec 4:30 p.m. PST — 7:30 p.m. PST [Knowledgeable Agents] Agent Planning with World Knowledge Model Location: East Exhibit Hall A-C #3311 Time: Thu 12 Dec 11 a.m. PST — 2 p.m. PST

Introducing our latest work, KnowAgent: Knowledge-Augmented Planning for LLM-Based Agents 🚀. Explore strategy for boosting the planning capabilities of LLM-based agents by leveraging external action knowledge base. #NLP #LLMs #Agent #KnowAgent #Planning #Knowledge 📌Project Page: zjunlp.github.io/project/Kno… 📖 Paper: arxiv.org/abs/2403.03101 🔗 Code: github.com/zjunlp/KnowAgent 🧩 Motivation: LLMs excel in complex reasoning, yet occasionally encounter "planning hallucination"🚫👻, leading to the generation of unnecessary or conflicting actions. Such issues diminish their efficiency and effectiveness in real-world applications. 💡Solution Overview: KnowAgent enhances the planning process through a systematic incorporation of explicit action knowledge with these key steps: 1️⃣Action Knowledge Base: This compiles essential action planning knowledge relevant to specific tasks, serving as an informational guide to direct the model's action generation efforts. 2️⃣Path Generation: By converting action knowledge into text, we enable the model to understand and utilize this knowledge in creating action trajectories. 3️⃣Knowledgeable Self-Learning: Guides LLMs to learn and refine action paths, ensuring more effective and accurate planning. 🔬Results: Our experiments with models on HotpotQA and ALFWorld demonstrate KnowAgent's capacity to match or exceed existing baselines, reducing planning hallucinations. We're excited to hear your thoughts and feedback ! 🎙️

🤔Can NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models? Excited to collaborate with @zjuzjt @dsmall2apple1 @BryanHooi1 @RuiboLiu @IreneXuPiano ! 🚀 We've updated our paper "Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View" with more extensive models (e.g., LLaMA, Mixtral, Qwen) and experiments #NLP #LLMs #Collaboration #Agent #SocialPsychology. 📌Check out the project page: zjunlp.github.io/project/Mac… 📌Paper: arxiv.org/abs/2310.02124 📌Code: github.com/zjunlp/MachineSoM 🧐Some interesting findings from social psychology view (Not all findings are set in stone, just here for your consideration)： 1. Collaboration is generally effective in the group, especially for tackling difficult tasks. 2. Collaboration widely leads to conformity, either beneficial or harmful in performance. 3. As the number of rounds increases, benefits of conformity will decrease; and detriments of conformity will increase. 4. The totally easy-going society is more likely to reach a consensus, debate helps to consensus reaching while reflection impedes it. 📖Abstract: As Natural Language Processing (NLP) systems are increasingly employed in intricate social environments, a pressing query emerges: Can these NLP systems mirror human-esque collaborative intelligence, in a multi-agent society consisting of multiple large language models (LLMs)?  This paper probes the collaboration mechanisms among contemporary NLP systems by melding practical experiments with theoretical insights. We fabricate four unique 'societies' comprised of LLM agents, where each agent is characterized by a specific 'trait' (easy-going or  overconfident ) and engages in collaboration with a distinct 'thinking pattern' ( debate or  reflection). Through evaluating these multi-agent societies on three benchmark datasets, we discern that certain collaborative strategies not only outshine previous top-tier approaches, but also optimize efficiency (using fewer API tokens). Moreover, our results further illustrate that LLM agents manifest human-like social behaviors, such as conformity and consensus reaching, mirroring foundational social psychology theories. In conclusion, we integrate insights from social psychology to contextualize the collaboration of LLM agents, inspiring further investigations into the collaboration mechanism for LLMs. Welcome your valuable suggestions!

Unfortunately, some of the speakers including me cannot make it to Canada, we regret to cancel the tutorial "Knowledge Editing for Large Language Models" in accordance with #AAAI2024 's policy. Thanks for the support from the AAAI2024 Chairs @RealAAAI and everyone involved @Jiachen_Gu @dsmall2apple1 @Bizhen_Ray @yyzTodd. We've shared a 192-slide PPT for the community to check out. Stay tuned for more resources. 📌PPT: drive.google.com/file/d/1fkT… 📌A Comprehensive Analysis: arxiv.org/abs/2401.01286 📌KnowEdit Benchmark: huggingface.co/datasets/zjun…… 📌EasyEdit Tool: github.com/zjunlp/EasyEdit 📌Paper List: github.com/zjunlp/KnowledgeE…

Want to keep your AI agent up-to-date and aligned with your ideas? Try cutting-edge knowledge editing methods!🤖📷<✂️🔪 In #EMNLP2023, we presented the prior work “Editing Large Language Models: Problems, Methods, and Opportunities” for a review of LLMs editing techniques. After months of hard work, we have released the first version of our technical report of “A Comprehensive Study of Knowledge Editing for Large Language Models” for the community @yao_yunzhi @dsmall2apple1 @JasonForJoy @arxiv @ArXivGPT @arxiv . 🤔What's this all about? New Taxonomy. Based on the educational and cognitive research theories, we propose a new taxonomy to provide a unified view on knowledge editing methods: >Resorting to External Knowledge. This kind of approach is similar to the recognition phase in human cognitive processes, which needs to be exposed to the new knowledge within a relevant context, just as people first encounter new information. For example, providing sentences that illustrate a factual update as a demonstration of the model, allowing initial recognition of the knowledge to be edited (e.g., SERAC, IKE). >Merging Knowledge into the Model. This kind of approach closely resembles the association phrase in human cognitive processes, in which connections are formed between the new knowledge and existing knowledge in the model. Methods would combine or substitute the output or intermediate output with a learned knowledge representation (e.g., GRACE, MELO, CaliNet, T-Patcher). >Editing Intrinsic Knowledge. This kind of approach to knowledge editing is akin to the mastery phase in human cognitive processes. It involves the model fully integrating knowledge into its parameters by modifying the weights and utilizing them reliably (e.g., KE, MEND, ROME, MEMIT). Benchmark & Evaluation. We construct a new benchmark KnowEdit. We report the empirical results of cutting-edge knowledge editing approaches for LLMs, providing a fair comparison and illustrating their overall performance in the settings of knowledge insertion, modification, and erasure. We show the impact of knowledge editing on general tasks and multi-task knowledge editing, which implies that contemporary knowledge editing methods are effective in executing factual updates with minimal disruptions to the model’s cognitive capabilities and adaptability across diverse knowledge domains. Mechanism & Analysis. We summarize the mechanism of knowledge storage in LLMs and provide an analysis via knowledge editing. We observe a pronounced focus on one or several columns within the value layer in edited LLMs. Furthermore, we find that the process of knowledge locating (e.g., casual analysis) tends to pinpoint only the areas related to the entity in question, rather than the entire factual context, suggesting that LLMs might be deriving answers either by recalling information memorized from their pre-training corpus or through a multi-step reasoning process. Additionally, we delve into the possibility that knowledge editing for LLMs could lead to unintended consequences, an aspect warranting careful consideration. Applications. We delve into the applications of knowledge editing, examining its potential from a variety of perspectives, including efficient machine learning, AI-Generated Content (AIGC), trustworthy AI, and human-computer interaction (personalized agents): >Efficient machine learning. Update the LLMs for new task, domains or fixing bugs, and manipulate the LLMs to prune or merge parameters for new abilities efficiently. >AI-Generated Content (AIGC). Utilize knowledge editing methods to modify/control the content (e.g., text, images) generated by LLMs. >Trustworthy AI. Leverage knowledge editing techniques for detoxifying, debasing, and defense strategies for privacy issues. >Human-computer interaction (personalized agents). Control LLMs to serve as personalized agents, and harness the capabilities for social good. We will continue to update this paper. Tackling a tough challenge here, with many unresolved questions. Open to insights and corrections - chime in! #Discussion #FeedbackWelcome #NLP #LLMs #ModelEditing #KnowledgeEditing 📌ArXiv: arxiv.org/abs/2401.01286 📌KnowEdit Benchmark: huggingface.co/datasets/zjun… 📌EasyEdit Tool： github.com/zjunlp/EasyEdit 📌Paper List: github.com/zjunlp/KnowledgeE…

A Comprehensive Study of Knowledge Editing for Large Language Models paper page: huggingface.co/papers/2401.0… Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication. However, a primary limitation lies in the significant computational demands during training, arising from their extensive parameterization. This challenge is further intensified by the dynamic nature of the world, necessitating frequent updates to LLMs to correct outdated information or integrate new knowledge, thereby ensuring their continued relevance. Note that many applications demand continual model adjustments post-training to address deficiencies or undesirable behaviors. There is an increasing interest in efficient, lightweight methods for on-the-fly model modifications. To this end, recent years have seen a burgeoning in the techniques of knowledge editing for LLMs, which aim to efficiently modify LLMs' behaviors within specific domains while preserving overall performance across various inputs. In this paper, we first define the knowledge editing problem and then provide a comprehensive review of cutting-edge approaches. Drawing inspiration from educational and cognitive research theories, we propose a unified categorization criterion that classifies knowledge editing methods into three groups: resorting to external knowledge, merging knowledge into the model, and editing intrinsic knowledge. Furthermore, we introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches. Additionally, we provide an in-depth analysis of knowledge location, which can provide a deeper understanding of the knowledge structures inherent within LLMs. Finally, we discuss several potential applications of knowledge editing, outlining its broad and impactful implications.

New work with ByteDance, Stanford, and UC Berkeley, focus on benchmarking and improving LLM-powered multi-agent. We also introduce Bayesian method into LLMs,significantly enhancing LLM by an average of 50%. More details can be found on project page: zhiyuanhubj.github.io/MAgIC/

📄 "Towards A Unified View of Answer Calibration for Multi-Step Reasoning" 🤖 We're diving deep into the world of #AnswerCalibration in #MultiStepReasoning via prompting #LLMs. Discover a unified view to establish connections between different answer calibration strategies including step-level and path-level calibration @dsmall2apple1 .🧠💡 🔗 ArXiv: arxiv.org/abs/2311.09101 #AIResearch #COTprompting #MachineLearning #NLP #LLM #Reasoning

Excited to chat about “Knowledge Editing for Large Language Models" at #AAAI24 in our tutorial with Jiachen Gu, Yunzhi Yao, Zhen Bi and Shumin Deng! Thanks a lot to the organizer. Dive deeper into the topic with resources on github.com/zjunlp/KnowledgeE…. @RealAAAI #LanguageModels #Tutorial #NLP #LLM #ModelEditing #KnowledgeEditing #EasyEdit #KnowLM Related paper: arxiv.org/abs/2305.13172 Tools: github.com/zjunlp/EasyEdit

IJCNLP-AACL 2023 has accepted exciting tutorials! Check out the list! afnlp.org/conferences/ijcnlp… #AACL2023 #NLProc