돌 retweeted
아 꿈을 꾼다는 건 평소에는 함께 잘 쓰이지 않는 뇌의 회로를 랜덤하게 깨워서 전혀 다른 지식 영역이 엉뚱하게 섞이면서 창의적인 연결 고리와 패턴을 찾는 시뮬레이션 과정인거야?
Today on the blog, learn about our new instance of a “sleep” phase in continual learning, which enables LLMs to continuously integrate new facts and enhance reasoning while mitigating catastrophic forgetting or static knowledge cutoffs.
More →goo.gle/4iICkyD
ALT Overview of our sleep framework: Input streams from the active phase are stored in short-term memory before entering the sleep cycle, which bifurcates into Consolidation (transferring fragile short-term tokens into expanded long-term parametric memory using low-rank structures and MoE) and Dreaming (autonomous self-improvement via synthetic routing with random experts and RL refinement).
