engineer
San Francisco, CA
Joined September 2009
- Tweets580
- Following356
- Followers5.1K
- Likes2.1K
Richard Wei retweeted
Today we also published benchmarks and a deep dive into LLM inference on M5 Ultra Mac Studio, covering MLX, GPU Neural Accelerators, tensor and pipeline parallelism, and Thunderbolt 5 RDMA.
machinelearning.apple.com/re…
Muse Spark Open -- weights releasing today -- runs great on new Mac Studios thanks to work with @johnternus and Apple.
Richard Wei retweeted
You can now use @OpenAIDevs ‘s Decision model via Apple’s Foundation Model framework!
One API to rule them all 💍
Richard Wei retweeted
Decision models like Jev are all the rage - they are super fast, dirt cheap, and - best of all - deterministic.
What if... you could use Jev (and other System One models) using Apple Foundation Models framework?
Meet System One models in Swift:
go.peterfriese.dev/system-on…
Richard Wei retweeted
Want to use @typesafeai's Jev in your @SwiftLang apps using the convenience of Apple's Foundation Models framework?
Well - here you go (GitHub repo link in the replies)!
(h/t to @rxwei for the inspiration!)
Richard Wei retweeted
I've been using the on-device Apple foundation model to test out the SoC performance on the iPhone 17 Pro Max and 18 Pro Max
A19 Pro
Long Prompt: 2340ms TTFT, 45 tok/s
Medium Prompt: 946ms TTFT, 70 tok/s
Short Prompt: 638ms TTFT, 100 tok/s
A20 Pro
Long Prompt: 1125ms TTFT, 55 tok/s
Medium Prompt: 437ms TTFT, 71 tok/s
Short Prompt: 354ms TTFT, 150 tok/s
Richard Wei retweeted
With iOS 27 Apple introduced its most powerful on-device model: AFM 3 Core Advanced
It’s a 20 billion parameters model activating only between 1 to 4 billion parameters depending on the request
Here’s the model running on iPhone 17 Pro
Richard Wei retweeted
At our WWDC26 Recap event this week, I shared an overview of bringing intelligence into your apps on Apple platforms.
1/ Foundation Models Framework: Native Swift API for on-device, PCC & third-party LLMs (e.g. @GoogleDeepMind gemini, @claudeai, and others.)
2/ Core AI: Purpose-built framework for bringing custom models into your apps on Apple Silicon
3/ MLX: Open-source framework for model research, fine-tuning, local inference servers, and distributed computing
4/ Agentic Experiences: Multi-model workflows and context handoffs with specialized agents
5/ Evaluations: Automated tools to measure quality, uncover failures, and help you ship with confidence
Watch: youtube.com/live/V2i8f_NeKDI…
Richard Wei retweeted
The overview maps everything else, including the bit nobody mentions: macOS 27 ships an fm CLI, and fm serve gives you a local OpenAI-compatible endpoint backed by Apple's models. I'll update the post as I publish the rest: ivanmagda.dev/posts/wwdc26-f…
Richard Wei retweeted
My favorite part: Dynamic Profiles. One session swaps instructions, tools, and the model mid-conversation via a SwiftUI-style body. I built these mechanisms by hand for my coding-agent series, so I mapped Apple's versions onto ours, footguns included: ivanmagda.dev/posts/foundati…
Richard Wei retweeted
PrivateCloudComputeLanguageModel: a 32K-context server model with reasoning, switched on with one line. No account, no keys, no per-token bill for eligible apps. Each user spends from a daily quota instead, and that quota deserves real UI. Deep dive: ivanmagda.dev/posts/foundati…
Richard Wei retweeted
Last June, Foundation Models was a Swift API to one small on-device model. After WWDC26 the same session API runs Apple's server model, Claude, Gemini, or a local Qwen. I spent the week in the betas. Three posts on what changed:
Richard Wei retweeted
We had a blast at the Machine Learning & AI Group Lab today! ML technologies, frameworks, models, evaluations, and lots of fun with @trebolloc, @LouisDhauwe and others.
🎬 Video is up on YouTube - questions and discussions on Core AI, MLX, Foundation Models framework, Evaluations and more!
WWDC26 was amazing for AI/ML. Super excited for all the incredible experiences you’ll build with these technologies! Thank you!
youtube.com/live/n6TdbD7nh2k…
Richard Wei retweeted
You can now use Gemini via Apple's Foundation Model framework.
This means:
- same calls for on-device and cloud-hosted inference
- you can use Grounding with Google Search / Google Maps
- access to Gemini's large context window (up to 1M tokens)
Richard Wei retweeted
Foundation Modelsめっちゃ進化してる😲iOS 27では、オンデバイスモデルにVision機能が追加されたことにより、画像に関する質問にも答えられるように。
UIImage, NSImage, CGImage, Core Image 型, CoreVideo, Pixel Buffer, ファイル URLの形式に対応。
大きな画像ほどトークンを消費する #wwdc26
Richard Wei retweeted
If you missed our AI/ML Group Lab yesterday, we're doing another one on Friday June 12, 2026 9:00 AM – 10:00 AM PT. We'll discuss Core AI, Foundation Models framework, Evaluations framework and MLX. Join us!
developer.apple.com/wwdc26/s…
We wrapped up an amazing panel discussion on the latest in AI and Machine Learning at WWDC26. Thank you for all your questions about agents in Xcode, Core AI, Evaluations framework, Foundation Models framework, MLX. Here is the recording ICYMI.
youtube.com/live/bXb18GwYQS8…
Richard Wei retweeted
For WWDC26, Apple has introduced a shift in how you think about the foundation models framework.
The broad "Foundation Models" term could now refer to any of these 3 things, which can be confusing.
→ Apple Foundation Models (AFM) - the actual on- device and server LLMs from Apple .
→ Foundation Models Framework - a model-agnostic layer for running LLMs on Apple devices (both local AND server, Apple AND Third-Party) .
→ Foundation Models Framework Utilities - An open source package that adds extra utilities for working with LLMs. Supported on apple platforms and linux.
The Framework is now the main product. AFM is just one of the options you can plug in.
Richard Wei retweeted
Anthropic and Google recently announced support for Apple's Foundation Models framework, making state-of-the-art Claude and Gemini models available to all Swift developers.
claude.com/blog/claude-for-f…
blog.google/innovation-and-a…
Richard Wei retweeted
Wait, what? 😱
Apple Foundation Model on Private Cloud Compute is working on fm CLI!
Here's a video!
Richard Wei retweeted
One of my favorite features in macOS 27 is fm CLI. Simply open the terminal and type:
$ fm chat
And start chatting with the on-device or Private Cloud Compute (PCC) model. You can also use fm in your bash scripts to automate tasks! Watch it in action:
youtu.be/pvefPi4J2G4?t=239&s…