ML Researcher | Core contributor to Apples official ML framework (MLX)| Violin enthusiast 🎻 | Always coding, sometimes watching Anime.
Germany
Joined February 2022
- Tweets2.2K
- Following876
- Followers3.2K
- Likes68.7K
Pinned Tweet
My name got featured in one of Apples WWDC 2025 session videos 👀🍎
Crazy to think my work made it into a global Apple dev session. I’m proud!!
Thanks @awnihannun @angeloskath @Apple
Finishing up the last two PRs for base VLM training for MLX-VLM by @Prince_Canuma to then start adding decision model training. And then………
Here are the current PRs:
Multi-Token-Prediction: github.com/Blaizzy/mlx-vlm/p…
Quantization Aware Training: github.com/Blaizzy/mlx-vlm/p…
Open questions I have for the new Full Duplex Dessa model:
- Is it only pertained on audio? If so, then how can we fine-tune it? Or will there be fine-tuned models released along side it?
- How can we add async tool calling into it when it was trained on audio only?
Just tell Codex to train a custom model for you on apple silicon!!!!!
Here is a real example of codex pre-training a tiny LLM and iterating on it to get the best results, using MLX-LM-LoRA's MCP part.
How to reproduce it:
1. Install mlx-lm-lora: python3 -m pip install -U "mlx-lm-lora[mcp]"
2. Install the skill: npx skills add Goekdeniz-Guelmez/mlx-lm-lora --skill mlx-lm-lora -a codex -g
3. Find out what path the mcp package is: which mlx-lm-lora-mcp
4. Copy the path and copy it into here: codex mcp add mlx-lm-lora -- {path}
5. Start chatting
Here is the prompt I used:
"""Task: Find the best pre-training run for tiny-random/qwen3
Use the mlx-lm-lora MCP server and skills only. No other training stack.
Fixed
Model: tiny-random/qwen3 (if it doesn't resolve, ask me before continuing)
Dataset: mlx-community/recycling_the_web-1k
Task: pre-training
Steps: 100
Global batch size: 8
Context length: 128
Do
Run a single SFT pre-training recipe 4 times, letting you pick the hyperparameters and optimizer trial after trial yourself. Keep the fixed constraints identical every run. Use a different seed per trial and evaluate each on a held-out slice.
Deliver
A table of the 4 trials: optimizer, lr, schedule, seed, final train loss, val loss/perplexity.
The best pre-training trial by val loss, with its config and saved adapter path.
If any fixed constraint can't be satisfied by the MCP server, stop and tell me."""
I've gotten a lot of comments saying it's just my FYP algorithm feeding me more voice AI content, but I don't think that's true here. Realistically is probably a massive coincidence, with all of us working toward similar goals at the same time.
But just to look at this timeline:
October 18 - First JOSIE-Voice teaser video
October 20 - Mistral Voxtral 3 + ElevenLabs Eleven Realtime v1 + Kyutai Pocket TTS v4.0.0
October 21 - Gradium Phonon TTS release
October 22 - IBM Granite Speech 5.0 + Amazon Polly Generative
October 23 - Block Dessa full-duplex voice model teaser
Within the same week as my JOSIE-Voice teaser, multiple teams released technology directly relevant to building efficient, real-time, local, voice agents. I'm not saying there's a conspiracy here. I'm just saying the timing is getting suspiciously cinematic. 🤔
jk, its probably just coincidence 😂😂😂
Gökdeniz Gülmez retweeted
Calling all AI safety researchers.
We’re inviting a small group to explore Dessa, our model trained on speech alone, and help us understand what it can and can’t do.
We’d love to hear from you: block.xyz/dessa
Another real world 6 minute voice conversation (with a lot of interruptions, on the fly steering style mind changes and Apple Notes integration) with JOSIE-Voice using MLX-Audio-Swift's Kokoro and Qwen3 ASR backend fully local on apple silicon. This conversation is about Dessa!!!!!
P.S. Don't know why the video is so quiet 😂😂
Meet Dessa, our AI model trained from scratch on nothing but speech.
We believe speech is the natural interface for technology. So we built Dessa.
And we’re inviting AI safety researchers to explore Dessa.
Learn more: block.xyz/dessa
Gökdeniz Gülmez retweeted
Flint and Phonon now run in Nativ. Local coding + on-device voice, all on your Mac 📷
Nativ v0.4.3:
→ Recursive Flint: 40B MoE, day 0 on Apple Silicon
→ Gradium Phonon TTS: 4 voices, first audio under 15 ms
→ New Phonon deep dive on the blog
→ Smoother model picker
→ Snappier chat sidebar
→ Free local AI. Nothing leaves your Mac
Try Nativ: blaizzy.github.io/nativ
Gökdeniz Gülmez retweeted
The most exciting speech to speech model to date 🔥
STS is really close to my heart, after working on the first modular STS pipeline for MLX-Audio nearly 2 years ago I learned that the bottleneck was a the model.
Now we have a breakthrough that will unlock it
Gökdeniz Gülmez retweeted
Local AI Builders Friday #1 🦙
On Oct 10, I highlighted 24 builders from my Local AI Builders list. The list is now 94!
Starting this week, as part of this series, every Friday I'll shout out new Local AI people and the ones I've missed.
So many OGs and so many cool people are building at the intersection of Local AI, MLX, open source AI, open-weight models and quantization.
Worth a follow:
- @hnshah
- @gerdelgado
- @ClementDelangue
- @julien_c
- @TeksEdge
- @bleysg
- @Teknium
- @haider1
- @HessianFree
- @SteveSTechMan
- @u1tra_instinct
- @mervenoyann
- @mchiang0610
- @wightmanr
- @alexocheema
- @thebasedcapital
- @VikParuchuri
- @_philschmid
- @turboderp_
- @natolambert
- @shshnkp
- @macintogdev
- @ramin_m_h
- @0xSigil
- @theemozilla
- @digitalix
- @goodhunt
- @Technovangelist
- @Hikari_07_jp
- @code_hiyouga
- @tcarambat
- @_LuoFuli
- @zhuohan123
- @ActuallyIsaak
- @gladkos
- @philipkiely
- @ornith_
- @xenovacom
- @MaziyarPanahi
- @SahinLale
- @lgtm_hbu
- @sudoingX
- @jartine
- @woosuk_k
- @jeremyphoward
- @stevibe
- @StasBekman
- @FrostForger
- @DanKornas
- @melvindvivas
Pin it and you get a feed that's only Local AI.
Full list: xitter.cf/i/lists/21089799789857…
Who else should I add?
I JUST SET UP A MAC MINI TO RUN AI LOCALLY
M5 Pro, 64GB of memory. First six models, measured with Apple's MLX, in tok/s:
- Qwen3 30B at 106
- Qwen3 8B at 61
- Laguna XS 2.1 at 56
- Qwen3 14B at 34
- Ternary Bonsai 2 27B at 26
- Gemma 3 27B at 18
Models from @Alibaba_Qwen, @poolsideai, @GoogleDeepMind (@googlegemma) and @PrismML.
Qwen3 30B and Laguna XS 2.1 only use about 3B of their parameters for each token, so they run like small models.
Ternary Bonsai 2 27B stores nearly every weight as -1, 0 or 1. It fits in 8.4GB, about half what Gemma 3 27B needs.
When I'm not using it, the Mac serves open-weight models on @darkbloomai, the compute grid powered by people's Macs. (I work at @eigenlabs which builds it)
I strongly believe it matters whether people rent AI or own it. A model on your own machine can't be taken away or quietly changed. You can run it, take it apart and build on it without asking anyone's permission.
Open-weight models and open source AI make that freedom real and as AI shapes more of how we work and think, it's worth protecting.
I've also installed @ollama and @lmstudio for chatting with models in an app and @tailscale so I can reach the Mac from anywhere
I'm not an engineer. I'm learning in public and logging everything in a repo. (check first reply)
Local AI folks I keep learning from, who share what they figure out in public:
- @ggerganov
- @awnihannun
- @danielhanchen
- @jpschroeder
- @ivanfioravanti
- @MichaelGannotti
- @Tim_Dettmers
- @simonw
- @Prince_Canuma
- @rasbt
- @maximelabonne
- @TheAhmadOsman
- @QuixiAI
- @loktar00
- @itsjustmarky
- @jmorgan
- @ddalcu
- @GumbiiDigital
- @MikeBradleyAI
- @StefanMaier
- @ashen_one
- @TheDavidTai
- @Spangler3000
- @ashxhart
Who else should I follow? I'm adding everyone to my Local AI Builders list (link below).
Gökdeniz Gülmez retweeted
JOSIE-Voice is a real-time voice agent from @ActuallyIsaak, with a 2.5-minute real call to show it off.
xitter.cf/ActuallyIsaak/status/2…
trying this out on codex + MLX-LM-LoRA on @OpenAI parameter golf challenge!
github.com/openai/parameter-…
Introducing MLX-LM-LoRA v5.9.0
This brings LLM training to your AI agent through MCP.
Describe what you want in plain language. Your agent can validate the training config, launch a fine-tune on Apple Silicon, monitor progress, inspect logs, and stop the run.
The goal: make improving an LLM something you can ask for in a conversation.........or automate.
This is a first step toward automating the improvement loop aka RSI for everyone: evaluate → train → evaluate → iterate.
How to use it:
python3 -m pip install -U "mlx-lm-lora[mcp]"
Codex: codex mcp add mlx-lm-lora -- /full/path/to/mlx-lm-lora-mcp
Claude-Code: claude mcp add --transport stdio --scope user mlx-lm-lora -- /full/path/to/mlx-lm-lora-mcp
Install the skill: npx skills add Goekdeniz-Guelmez/mlx-lm-lora --skill mlx-lm-lora -a codex -a claude-code -g
github.com/Goekdeniz-Guelmez…
Gökdeniz Gülmez retweeted
Coming to MLX -VLM and @Nativ_AI 🚀
MiniMax M3.1 Flash weights are now public. 428B parameters total, 23B activated per token.
Agentic performance: A generational leap in agentic tasks, coding, and cybersecurity — 76.2 on BankerToolBench, 73.7% on DeepSWE v1.1, 61.7 on OSWorld V2, 29.6 on Agents' Last Exam, and 82.2 on CyberGym.
Efficient by design: MiniMax Sparse Attention and DSpark speculative decoding improve inference efficiency.
Hugging Face:
huggingface.co/MiniMaxAI/Min…
ModelScope:
modelscope.cn/models/MiniMax…
API: platform.minimax.io
M Plan: minimax.io/m-planMiniMax
MCode for desktop: agent.minimax.io/download
Introducing MLX-LM-LoRA v5.9.0
This brings LLM training to your AI agent through MCP.
Describe what you want in plain language. Your agent can validate the training config, launch a fine-tune on Apple Silicon, monitor progress, inspect logs, and stop the run.
The goal: make improving an LLM something you can ask for in a conversation.........or automate.
This is a first step toward automating the improvement loop aka RSI for everyone: evaluate → train → evaluate → iterate.
How to use it:
python3 -m pip install -U "mlx-lm-lora[mcp]"
Codex: codex mcp add mlx-lm-lora -- /full/path/to/mlx-lm-lora-mcp
Claude-Code: claude mcp add --transport stdio --scope user mlx-lm-lora -- /full/path/to/mlx-lm-lora-mcp
Install the skill: npx skills add Goekdeniz-Guelmez/mlx-lm-lora --skill mlx-lm-lora -a codex -a claude-code -g
github.com/Goekdeniz-Guelmez…
Why did all of a sudden so many TTS and STT model get released right after I revealed JOSIE-Voice????????????
Gökdeniz Gülmez retweeted
Congrats to @RichardSocher and the whole @Recursive_SI team on the release!
Flint comes with Day-0 support on @Nativ_AI and MLX-VLM
Runs at 419 tok/s at bach of 8 on a M3 Ultra, 512GB.
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Gökdeniz Gülmez retweeted
Introducing a new identity for Apple Music, physically shaped by an original composition from @fredagainagain1 and NEST Acoustics.
Gökdeniz Gülmez retweeted
We’ve partnered with @Recursive_SI to bring day-0 support for Flint, their first OSS model, to Apple Silicon with @Nativ_AI and mlx-vlm 🚀
Key features:
→ 40B params, 2.5B active
→ Apache 2.0
→ Runs 100% local on Apple Silicon
🧵
Phonon is a great TTS model and as they said its perfect for real time voice agents like JOSIE-Voice, here is a real world call where I switched to the multilingual TTS model, it sounds great, but has some funny pronunciations 😂😂
A new frontier for on-device speech.
We're open-sourcing Phonon, a suite of TTS models under 50m parameters, setting a new standard for naturalness and accuracy across languages, with the broadest inference support to date.
Phonon-31m is available under Apache 2.0.
gradium.ai/phonon🧵