Patrick Devine (@pdev110) profile picture

Software Guy @ Ollama

San Francisco Bay Area
Joined May 2014
Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)Photo by Patrick Devine (@pdev110)
For your pandemic Friday viewing enjoyment. 'kubectl run -it --rm --image=ghcr.io/pdevine/thisisfine tif' or for the docker inclined 'docker run -it --rm ghcr.io/pdevine/thisisfine'
2
1
1
9
More @thinkymachines inkling-2-mini running in @ollama w/ @NousResearch Hermes CLI. I just love that it was able to take screenshots of its progress and show it in the harness.
1
7
431
Playing around with inkling-2-mini on @ollama v0.40.17-rc3. It's pretty dope so far and runs well on a 32GB Mac. This is my standard "make an html5/js asteroids game so colourful that it makes my eyes bleed" prompt.
1
2
14
3,288
Patrick Devine retweeted
.@GoogleDeepMind EmbeddingGemma 2 is now available on Ollama. Made for consumer devices and is multimodal. ollama pull embeddinggemma-2
Meet EmbeddingGemma 2, our first natively multimodal open model for on-device embeddings. It expands beyond text to unify code, images, audio, and video in a shared space. 🧵
25
49
5
669
35,623
Patrick Devine retweeted
Ollama now supports Jev-like decision models all locally. Use decision models like Nimble for tasks like ticket triaging, model routing, and content moderation. ollama pull nimble Here’s Nimble playing Ollama racer through the new local /v1/systemone API by making decisions in real-time. 🏎️
175
605
122
4,586
669,205
Patrick Devine retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
3,372
12,212
2,709
71,653
9,651,293
Patrick Devine retweeted
Ollama v0.33 is here! You can now easily configure Claude Desktop to seamlessly work with Ollama as a third-party gateway provider. One toggle. Cloud & local models just work👇
120
243
50
2,580
497,366
Patrick Devine retweeted
Got Meta's new Muse Glimmer 30B running on my MacBook (M3 Max, 96GG) and tested the serving options available so far. Fastest right now: Ollama's MLX engine (DFlash included) at ~29 tok/s. Tuned llama.cpp: ~21. Raw mlx-vlm: ~10, not optimized yet. Numbers below if you're setting it up 👇
6
8
43
9,041
Patrick Devine retweeted
Currently serving 200 tps+ output speed for DeepSeek-V4-Flash on Ollama's cloud with zero data retention (ZDR). Have an amazing weekend 🫡
DeepSeek-V4-Flash-0731 is now fully rolled out as the new default for deepseek-v4-flash on Ollama's cloud. This model combines speed, efficiency, and frontier-level performance. Fast: 120+ output tps on Ollama's cloud Private: zero data retention hosting in US & Europe Efficient: generous usage on Ollama's Pro and Max plans for multiple long-running, uninterrupted sessions with your favorite coding harnesses.
96
91
20
2,047
189,892
Behold! Server grade RTX 6000 w/ a 3D printed shroud to strap on a cooling fan. This seems like a bad idea, but honestly running great so far. I'm wondering if there are good liquid cooling solutions though.
7
16
1,804
It's not as fancy as what @ivanfioravanti cooked up with the pagoda demo, but this was a single line prompt to make minecraft in godot using Kimi K3 on @ollama cloud.
2
1
13
1,166
19 hours in and still 5 hours to go.
1
155
Patrick Devine retweeted
Kimi K3 is now available on Ollama’s cloud. To use it with Claude Code, run: ollama launch claude --model kimi-k3:cloud Currently Kimi K3 requires a Pro or Max subscription, and consumes extra usage credits. We’re quickly working on adding capacity to expand access.
Releasing the model weights and technical report of Kimi K3. Kimi K3 is our most capable model: a 2.8T MoE model with native visual understanding and a 1M-token context window. New model architecture: 2.5x the intelligence per unit of compute, not just more params. Alongside Kimi K3, we're opening up more of the stack behind it — high-performance attention kernels, MoE communication library, and infrastructure for running agent environments at scale. Model weights: huggingface.co/moonshotai/Ki… Tech report: github.com/MoonshotAI/Kimi-K… Tech blog: kimi.com/blog/kimi-k3
Readers added context they thought people might want to know
Ollama is not including K3 inside any of the subscription plans, not even in the highest "Max" tier, it only has it as a PAYG model. ollama.com/library/kimi-k3
242
324
93
3,312
333,600
Patrick Devine retweeted
Ollama is proud to sign @satyanadella's letter. Our mission from day one has been to make open models accessible to every developer to unlock the next frontier in America and across the globe.
Open-weight models are essential to a healthy AI ecosystem. Together with others across our industry, we are outlining a path for open-weight models to strengthen American competitiveness and expand economic opportunity, while protecting national security. microsoft.com/en-us/corporat…
76
208
27
3,557
229,238
Patrick Devine retweeted
Big day for Ollama! When we started, open models and the open source AI ecosystem were in their early days with few believers. Our belief in open source has never wavered. With today's fundraising announcement and our 9M+ active builders, we’re ready to scale open models into AI that you can own. All aboard open models! 🧵
207
228
110
2,013
339,106
Lots of improvements here! Note that the nvfp4 e2b, e4b, 12b, and 31b models are all nvidia model optimized and should have lower perplexity. So more speed and better results. The 26b model is also updated, but we're still working some kinks out so more updates soon!
Gemma 4 is now nearly 90% faster on Apple Silicon with Ollama using MLX! The speedup comes from improved multi-token prediction (MTP), now on by default for Gemma 4, with more models to come. Ollama automatically tunes how many tokens to draft as it runs, so it never slows generation down when speculation no longer contributes to a speedup.
2
10
1,222
Patrick Devine retweeted
Banning or restricting Chinese models in the US would backfire and cede the global AI ecosystem to Beijing the new AI paradox: slowing America, speeding China
87
122
38
767
251,142
Patrick Devine retweeted
GLM 5.2 on Ollama's cloud just doubled GPU capacity to handle the volume of usage! This is all US based, and running on NVIDIA B300 Blackwell GPUs. We believe privacy matters! Let's go open models! ❤️
142
286
54
5,147
540,545
Patrick Devine retweeted
Gemma 4 Quantization-Aware Training (QAT) weights are now available on Ollama! They reduce memory requirements while maintaining model quality. E2B: ollama run gemma4:e2b-it-qat E4B: ollama run gemma4:e4b-it-qat 12B: ollama run gemma4:12b-it-qat 26B: ollama run gemma4:26b-a4b-it-qat 31B: ollama run gemma4:31b-it-qat Try them with ollama launch integrations to use with your favorite tools 👇👇👇
We just dropped Gemma 4 Quantization-Aware Training (QAT) checkpoints on Hugging Face! All Gemma 4 model sizes and their drafters are now optimized with QAT to cut memory requirements and maximize on-device performance!
42
153
14
1,409
112,566
I think almost all of the MTP/DFlash demos I've been seeing over the last few weeks have been using simple greedy sampling. That's great if you can live with temperature = 0, but I think most people want more sampling options.
145