
Building Causal Reflection @based_models — boost open models to frontier reliability without fine-tuning or RL | ML Engineer & Researcher | Stanford
CA
Joined June 2017
- Tweets70
- Following297
- Followers100
- Likes730
Zac Liu retweeted
General agents have mastered code. Video is next.
I quit my computer vision research job at Meta AI to spend two years building a video-first agent harness. Coding agents made building easy, so launching is now 10× more competitive.
Meet AdaL v2, the video agent for GTM.
• Generate and edit video, images and audio entirely through chat
• Version control to explore ideas and go back to the one you like
• Multiple videos in one project, from your main launch film to formats for each social platform
HyperFrames and Higgsfield MCPs are add-ons inside Claude Code. AdaL v2 is our full coding agent harness, optimized natively for video, with its own desktop app. Compared with them:
• 10× more control: you see and direct the whole production, not unreadable bash commands
• 10× faster iteration: you reuse and reproduce any asset from past versions
• 10× easier to use: one plan for video, image and audio models, zero setup, with browser use and deep research built in
Launch films, product demos, explainers, UGC ads, every platform cut. Launching just got easy again.
We’re opening a limited beta of the Model API (it maintains a lightweight causal graph for your sessions, visible in the dashboard).
Waitlist: forms.gle/iVHZjQbzAWSaHCyG8
Blog: basedmodels.com/blog/introdu…
1
64
Zac Liu retweeted
LLMs are poor reasoners, everyone knows this. But most never ask why?
A seminal critique comes from Mirzadeh et al. (2024) in their paper "GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models" (Apple Machine Learning Research).
The authors extended the GSM8K benchmark by introducing symbolic variations (e.g., altering names, numerical values, or adding irrelevant clauses) while preserving the underlying logical structure.
They found that state-of-the-art models, including GPT-4o and o1-series, exhibited significant performance variance, drops of up to 65% when irrelevant but seemingly plausible clauses were added concluding : "We found no evidence of formal reasoning in language models. Their behavior is better explained by sophisticated pattern matching so fragile, in fact, that changing names can alter results by ~10%."
This fragility is also echoed in Dziri et al. (2024) from MIT CSAIL, titled "Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks". By testing LLMs on counterfactual variants of tasks (e.g., altered chess positions or musical fingering), the authors observed consistent performance collapses in novel scenarios, attributing success on standard benchmarks to memorization rather than robust reasoning.
Techniques like Chain-of-Thought (CoT), Tree-of-Thought (ToT), or hierarchical reasoning (e.g., planning + execution) improve performance by guiding models to generate intermediate steps, mimicking human deliberation.
These are not true reasoning because:
1. Gains often stem from providing better context or "magic keywords" that align attention to relevant patterns, not internal logical processes.
2. Recent analyses describe CoT as a "mirage" models produce plausible-looking chains post-hoc, but these may not faithfully represent how the answer was derived (or could be illogical in smaller models).
3. Performance remains brittle: Models hallucinate, fixate on superficial cues, or degrade with depth/complexity.
4. Even advanced "reasoning models" (e.g., o1 series, DeepSeek-R1) scale inference-time compute for longer chains, yielding impressive benchmarks, but underlying failures persist in out-of-distribution or adversarial cases.
5. Most importantly, self-correction often fails or worsens errors without external feedback.
In essence, these frameworks scaffold pattern matching into better approximations of reasoning, but don't instill systematic abstraction, causal understanding, or reliable generalization. LLMs "recite" trained paths more effectively, not derive logic. So excellent for compression i.e. knowledge retrieval but awful for reasoning i.e. decision making.
Zac Liu retweeted
No one teaches this, but this is what really happens when you hit `run` on an LLM.
User → API → Engines → Multi-GPU → CUDA → Hardware
I mapped every layer (100+ components) of the LLM Inference Stack so you can finally see the full picture.
Full blogpost coming soon!
Zac Liu retweeted
The 2023 @Kaggle AI report is out! I was honored to be part of this project, and I served as the area chair for Tabular and time series data. There are many incredible insights in this report, and I urge you all to take a look at it. Here are some links:
Kaggle AI Report Essay Competition: kaggle.com/competitions/2023…
pdf version of the 2023 Kaggle AI report: storage.googleapis.com/kaggl…
Slides for the report: kaggle.com/AI-Report-2023
#AI #ArtificialIntelligence #MachineLearning #ML
Anyone interested in VC and early stage startups, check out @ventureco_op, a first-of-its kind program by @LaconiaCapital that expands financial access into VC & sheds light on the inner workings of venture firms
Cohort 5 applications are open until 9/21
ventureco-op.medium.com/quin…
1
4
347
Zac Liu retweeted
wow. coming from @runwayml #Gen2 research.runwayml.com/gen2
While on the topic of video generation I was also mildy mind-blown a few days ago by multiControlNet and friends: reddit.com/r/StableDiffusion…
And the earlier, bit more professional take, "anime rock paper scissors": youtube.com/watch?v=GVT3WUa-…
The barrier to entry for creating animations/movies is evaporating quickly.
Would you watch this movie?
I made this intro using @runwayml and their new #Gen2 Txt2Video
the visuals in this clip are 100% created from Text Prompting!
#GenerativeAI is improving so fast. I'll continue my journey and share my result with y'all! #animation #art #ai #aiart
Zac Liu retweeted
Random note on k-Nearest Neighbor lookups on embeddings: in my experience much better results can be obtained by training SVMs instead. Not too widely known.
Short example:
github.com/karpathy/randomfu…
Works because SVM ranking considers the unique aspects of your query w.r.t. data.
Zac Liu retweeted
Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O device and tool specs, define the cognitive loop, page data in and out of context window, .run().
Massive Update for Auto-GPT: Code Execution! 🤖💻
Auto-GPT is now able to write it's own code using #gpt4 and execute python scripts!
This allows it to recursively debug, develop and self-improve... 🤯 👇
Zac Liu retweeted
🎉 GPT-4 is out!!
- 📈 it is incredible
- 👀 it is multimodal (can see)
- 😮 it is on trend w.r.t. scaling laws
- 🔥 it is deployed on ChatGPT Plus: chat.openai.com
- 📺 watch the developer demo livestream at 1pm: youtube.com/live/outcGtbnMuQ…
Announcing GPT-4, a large multimodal model, with our best-ever results on capabilities and alignment: openai.com/product/gpt-4
Zac Liu retweeted
There is a high disaster potential in the Banda Sea due to mega-thrust #earthquake and #tsunami threat. The region can produce Mw 8.8 or larger earthquakes, as in 1629. Unfortunately, this threat to the densely populated region is highly underestimated.
link.springer.com/article/10…
I will be presenting “AutoDC + AutoML = your AI dev superpower” at DataConLA 2022 (@DataConLA), with @TarlowScott
#dataconla #dataconference #datascience #artificialintelligence #machinelearning #datavisualization
or get 50% off when you register using this link. eventbrite.com/e/30306554620…
DataConLA is the largest data conference in Southern California and is celebrating its 10 year anniversary!!
You can register to Watch The Virtual Data Con LA for Free from here: dataconla.us12.list-manage.c…
1













