Abdalghani Abujabal retweeted
Language models handle a single question well. Keeping track of a whole conversation is harder, especially in languages like Arabic that are underrepresented in training data.
A user mentions early on that a medical condition stops them from flying. A few turns later they ask how to reach another country, and the model suggests a flight. The last question wasn't hard. The model just didn't carry forward what it had been told.
@scale_AI's MultiChallenge benchmark (Sirdeshmukh et al., 2025) was built to measure exactly this kind of failure. When it was released, every frontier model tested scored below 50%. Its data is in English, though, and an automatic translation wouldn't be a reliable Arabic test.
So, building on @scale_AI's framework, we created an Arabic version: 1,600 five-turn evaluation conversations in Modern Standard Arabic, covering 16 domains and 164 subdomains. It uses five challenge types:
1. Instruction Retention: does the model still follow an instruction from the first prompt?
2. Inference Memory: does it apply a detail the user mentioned earlier, even when the final request doesn't ask for it?
3. Reliable Versioned Editing: can it go back to an earlier version of a draft and edit that one?
4. Self-Coherence: does it stay consistent with what it said in earlier turns?
5. Contextual Anchoring (a new type we added): can it work out which topic the user means when they don't name it? For example, "the third architect we haven't talked about yet."
The full post includes one Arabic sample for each challenge type 👇
aiastrolabe.com/blog/multich…
Abdalghani Abujabal retweeted
Real users do not stop at a single prompt. They hold long conversations and ask models to work over long documents, and that is where things break.
A model can nail an isolated answer and still lose instructions, facts, or source constraints introduced 20 turns earlier.
We built an Arabic dataset to push exactly on that, spanning conversations up to 30 turns and long source documents, across Modern Standard Arabic and the Najdi Saudi dialect.
Two tracks:
- Long conversations: holding context and resolving references across dozens of turns.
- Document-grounded chats: staying faithful to a source, and saying "that's not in the document" instead of guessing.
Every conversation is written by native speakers and reviewed for factual accuracy, dialectal naturalness, source faithfulness, and cross-turn consistency.
We also tested three Arabic models on it, and they stumble on long, source-bound, dialectal conversations.
Read the full breakdown: aiastrolabe.com/blog/extreme…
Abdalghani Abujabal retweeted
AI conversations rarely happen in a single turn. Users ask follow-up questions, refer back to earlier information, introduce new requirements, and expect models to keep track of the conversation as it evolves.
Abdalghani Abujabal retweeted
Most Arabic AI demos look great, until real users show up.
Because users don’t speak in clean, structured Arabic. They speak in dialects, interrupt each other, talk over noise, and switch languages mid-sentence.
And this is where models break.
Abdalghani Abujabal retweeted
Can today’s AI models reliably read charts in Arabic? We tested them.
We ran a simple probe:
Two Arabic charts, two objective questions, tested across:
@OpenAI - GPT, @AnthropicAI - Claude, @Google - Gemini, @QatarComputing - @Ai_Fanar, and Z.ai - GLM-5
Abdalghani Abujabal retweeted
Incidents like this highlight a core problem we see every day:
agents behave differently once they touch real systems.
Testing, red-teaming, and alignment in realistic environments isn’t optional anymore - it’s foundational
Abdalghani Abujabal retweeted
Ramadan Kareem / رمضان كريم 🌙
As people and brands use AI to generate greeting images, it is critical that the images make sense linguistically and culturally.
Abdalghani Abujabal retweeted
In the recent paper Simple Test-Time Scaling: arxiv.org/pdf/2501.19393, researchers from @Stanford, @UW, @allen_ai , and @ContextualAI show that even 1,000 curated samples can significantly improve model performance.
Abdalghani Abujabal retweeted
We recently published a blog post redteaming leading LLMs on Arabic:
aiastrolabe.com/post/redteam…
Building on that assessment, we zoomed in on safe responses and categorized them further: (1) Helpful Response, and (2) Refusals.
Abdalghani Abujabal retweeted
Exciting news ahead! In Arabic, (ALLaM) means 'knowledge'. Get ready for our first launch of LLM, ALLaM - stay tuned for more updates soon.
(Tweet Written by ALLaM)
With - @areebsa @haidarkk1 @twairesh @ajabal4
Abdalghani Abujabal retweeted
Full interview :
youtube.com/watch?v=4idQbwsv…
Abdalghani Abujabal retweeted
Abdalghani Abujabal retweeted
في هذه الحلقة نتحدث مع @AbbadDira عن تجربته مع الطويلة مع @ArabicWikipedia. عبّاد يجيبنا هنا عن أسئلة تتكرر حول ويكيبيديا مثل من يديرها وينظم محتواها؟ وهل هي مصدر موثوق؟ من أين يبدأ المرء إذا أراد المساهمة في ويكيبيديا؟ وما الفوائد المرجوة من المساهمة؟
youtu.be/7kaqW0VLCFA
Abdalghani Abujabal retweeted
حوار شيق مع @DimaDamen من @BristolUni عن أبحاثها في مجال الرؤية الحاسوبية📺🤖 (#ComputerVision) و تعلم الآلة (#MachineLearning) و عن مسيرتها العلمية بما في ذلك اختيار التخصص في مراحل التعليم المختلفة و الفروقات بين أنظمة التعليم العربية و الغربية.
youtube.com/watch?v=tJBs0O8x…
Abdalghani Abujabal retweeted
حوار شيّق مع @Walid_Magdy من @EdinburghUni و @turinginst عن العلوم الاجتماعية الحاسبيّة (Computational Social Science)
youtube.com/watch?v=soKbFF-3…
#ArabicNLP #NLProc #ComputationalSocialScience
Abdalghani Abujabal retweeted
#بيتكوين و #Blockchain: ما هي #العملات_الرقمية و كيف تعمل؟
نتناول العملات الرقمية من خلال بيتكون و تقنية الـblockchain. المواضيع التي نعالجها تتضمن سبب اختراع هذه العملات و و كيف تعمل و مسألة الخصوصية و التزوير و هل تستثمر فيها؟ الحلقة كاملةً:
youtube.com/watch?v=mcctCKdm…
Abdalghani Abujabal retweeted
نقاشات في مختلف المجالات! 🧑🏻💻
#خليك_بالبيت واكتسب معلومات جديدة ←
youtube.com/channel/UCJdRnkp…
@elmnt14
Abdalghani Abujabal retweeted
في هذه الحلقة من #العنصر14 سنتحدث عن الـ #GPS، التقنية التي بفضلها توجد تطبيقات مثل كريم و أوبر🚖.كيف تعمل هذه التقنية🛰؟ و ما علاقة هذه الـGPS بالمواشي🐂؟
الحلقة كاملةً: youtu.be/-9JCufXziD8
لا تنسوا الاشتراك في القناة للمزيد! youtube.com/channel/UCJdRnkp…
Abdalghani Abujabal retweeted
في حلقة اليوم من #العنصر14 سنتحدث عن كيفية تحليل #البيانات لاكتشاف أنماطٍ مثيرةٍ للاهتمام، تساعد أصحاب الأعمال في تنمية أعمالهم من خلال فهمٍ أعمق لأنماط شراء زبائنهم🏬.
كما ويساعد أيضاً على تصميم أماكن البيع من الداخل 🛒.
مشاهدة ممتعة: youtu.be/ddIBz2_2S2k
#اكتشاف_الأنماط




















