Amalik Amriou (@amalik) header
Amalik Amriou (@amalik) profile picture

Data & AI Product Lead @OCTOTechnology - part of @Accenture - DeFi enthusiast - Irregular cyclist - Inconsistent runner

Fontainebleau, France
Joined November 2007
Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)Photo by Amalik Amriou (@amalik)
Amalik Amriou retweeted
Traditional RAG sucks because it promises "relevant chunks" but in fact returns "similar chunks". Relevancy requires reasoning. Introducing ReAG - Reasoning Augmented Generation
146
274
58
3,667
519,224
Amalik Amriou retweeted
How to Scale Your Model Google DeepMind just released an awesome book on scaling language models on TPUs. This is gold! Worth checking you are an LLM developer.
4
67
308
23,492
Language Models Use Trigonometry to Do Addition "We first discover that numbers are represented in these LLMs as a generalized helix, which is strongly causally implicated for the tasks of addition and subtraction, and is also causally relevant for integer division, multiplication, and modular arithmetic. We then propose that LLMs compute addition by manipulating this generalized helix using the “Clock” algorithm: to solve a + b, the helices for a and b are manipulated to produce the a + b answer helix which is then read out to model logits. "
55
334
49
2,338
195,306
Amalik Amriou retweeted
I wrote a short overview of DeepSeek R1's training process: thelmbook.com/articles/#!./D… Will follow-up with the knowledge distillation later this week.
14
133
7
746
46,085
Amalik Amriou retweeted
Woow a fully open source reasoning model on par with OpenAI o1 just released Deepseek R1 even outperforms Claude 3.5 Sonnet and o1-mini in almost all benchmarks. You can already use it for free (see below)
102
456
73
4,663
566,625
Amalik Amriou retweeted
⚠️ Developers, please be careful when installing Homebrew. Google is serving sponsored links to a Homebrew site clone that has a cURL command to malware. The URL for this site is one letter different than the official site.
237
2,392
375
10,774
1,357,638
Amalik Amriou retweeted
🌻kotaemon An open-source clean & customizable RAG UI for chatting with your documents Built with both end users and developers in mind. You can use off the shelf, or easily customize github.com/Cinnamon/kotaemon
11
108
3
525
63,608
Amalik Amriou retweeted
Optimizing LLM-based Multi-Agent Systems Presents a novel framework, OPTIMA, to enhance both communication efficiency and task effectiveness in LLM-based multi-agent systems through LLM training. Proposes an iterative generate, rank, select, and train paradigm with a reward function to improve performance, token use, and communication efficiency. Integrates Monte Carlo Tree Search-inspired techniques for DPO data generation to encourage diverse exploration. OPTIMA shows consistent improvements over single-agent baselines and vanilla MAS based on Llama 3 8B, with 2.8x performance gain with less than 10% tokens on tasks requiring heavy information exchange.
5
52
1
264
19,578
Amalik Amriou retweeted
Useful Prompting technique. Simply ask the LLM to re-read the question - significantly boosts LLM reasoning across diverse tasks and model types. 💡 Repeats question input twice in prompt, unlocks latent reasoning potential **Problem** 🤔: Decoder-only LLMs with unidirectional attention struggle with nuanced reasoning tasks due to limited global understanding of input questions. **Key Insights from this Paper 💡**: • Re-reading (RE2) input enhances reasoning by improving question comprehension • Enables "bidirectional" understanding in unidirectional LLMs • Compatible with existing thought-eliciting prompting methods • Effective across various LLM types and reasoning tasks **Solution in this Paper** 🔍: • Introduces RE2 (Re-Reading) prompting method: - Repeats question input twice in prompt - Enhances input understanding before reasoning - Allows tokens to attend to full context in second pass • Compatible with Chain-of-Thought and other prompting techniques • Applicable to zero-shot, few-shot, and self-consistency settings **Results** 📊: • Consistent improvements across 14 datasets and 112 experiments • Effective for both instruction-tuned (ChatGPT) and non-tuned (LLaMA) models • Increases n-gram recall between generation and input question • Most effective when reading question twice
36
161
25
1,037
141,434
Amalik Amriou retweeted
MemLong utilizes an external retriever for retrieving historical information which enhances the capabilities of long-context LLMs. It consistently outperforms other SoTA LLMs on long-context benchmarks and can extend the context length on a single 3090 GPU from 4k up to 80k.
8
73
1
305
27,329
Amalik Amriou retweeted
RAGEval Proposes a simple framework to automatically generate evaluation datasets to assess knowledge usage of different LLM under different scenarios. It defines a schema from seed documents and then generates diverse documents which leads to question-answering pairs. The QA pairs are based on both the articles and configurations. It seems that this benchmark can help with more reliably evaluating the knowledge usage ability of LLMs and avoids confusion regarding the source of knowledge (parameterized or retrieval).
4
104
3
490
76,179
Amalik Amriou retweeted
Direct Preference Optimization (DPO) has become one of the go-to methods to align large language models (LLMs) more closely with user preferences. If you want to learn how it works, I coded it from scratch: github.com/rasbt/LLMs-from-s…
18
303
7
1,649
168,208
Amalik Amriou retweeted
Few #Bitcoin traders understand this: If you want to long BTC with leverage. DON'T buy futures, buy spot BTC with margin. EXPLAINER (1) Buying futures can be fulfilled by any counter trader who has USD collateral, together you are minting new synthetic BTC to the supply which creates a bearish environment. (2) When buying spot with margin (borrowed USD), only BTC holders can sell it to you. This creates a supply shortage and a very bullish environment. BONUS In a bull market it's much cheaper to fund a long position with borrowed USD or USDT than with calendar futures or perps.
144
503
51
3,587
526,975
Amalik Amriou retweeted
Attention Instruction: Amplifying Attention in the Middle via Prompting Key findings: 1. LLMs lack relative position awareness 2. We can guide LLM to a specific region with position-based indexing Paper: arxiv.org/pdf/2406.17095 Thanks to: @mengzaiqiao and @nigelhcollier
4
33
3
117
11,528
Amalik Amriou retweeted
Cool paper proposing a graph-based agent system to enhance the long-context abilities of LLMs. It first structures long text into a graph (elements and facts) and employs an agent to explore the graph using predefined functions guided by a step-by-step rational plan. The agent accesses coarse graph components and detailed text, takes notes, and reflects until enough information has been gathered to generate an answer. This approach helps to effectively and reliably generate answers to questions. Claims to consistently outperform GPT-4-128k across context lengths from 16k to 256k. You can never sleep on the power of graph or tree structures which in this case helps to capture long-range dependencies and multi-hop relationships within long text. Similar to other tree structures I have reported in the past paper tweets, the agents now get to leverage enriched information to solve tasks.
4
78
6
349
28,620
Large Language Models Must Be Taught to Know What They Don't Know abs: arxiv.org/abs/2406.08391 Prompting is not enough for LLMs to produce accurate estimates of its uncertainty of its responses, but can be finetuned with as little as 1000 examples and outperform baselines for estimating LLM uncertainty.
9
129
7
717
74,898
My amalik@twitter Twitter handle can now receive crypto. #CRYPTO Get your own $FIO Crypto Handle for free now! dashboard.fioprotocol.io/twi… via @joinfio
1
81
I just voted in @SweatEconomy ’s governance vote in the #SweatWallet app! Join me here sweateconomy.com/download-wa… via @SweatEconomy
66
Amalik Amriou retweeted
Andreas Antonopoulos - 51% #Bitcoin Attack
120
1,535
328
3,429