Tianshu Yu (@TianshuYu6) header
Tianshu Yu (@TianshuYu6) profile picture

[email protected]. Do vision pretaining and architecture design for lfm3 vl. Democratize on-edge intelligence.

Joined July 2022
Photo by Tianshu Yu (@TianshuYu6)
Tianshu Yu retweeted
Today, we release d1.2 and d1.2-ultra, the state-of-the-art omnimodal decision models. d1.2 and d1.2-ultra both can process text, images, and audio with new output capabilities. > #1 on the @ArtificialAnlys Decision Index (75) > Read a value from the input (text, number or date) > Find every passage of a kind in a text (spans) > Locate things in a picture (box for one, boxes for every instance) > Estimate a number (number with a range) > Pick every option that applies (choice) 1/ ๐Ÿงต
210
329
117
3,696
25,609,777
Tianshu Yu retweeted
Adding vision to a language model is more than one extra modality. With large images or long video, the vision encoder can use more compute than the language model itself, so efficient models need a different design. Our VLM team, including @rshubert98, @TianshuYu6, @sina_semnani, and @iamsamstevens, explains how they teach LFMs to see, from mid-training to the custom vision encoder built for LFM3.
21
21
3
276
13,595
d1-2Bใ„ใ„ใฃใ™ใญใ‡ใ€œใ€‚jevใฏๅ…ƒใ€…ๅฎ‰ใ„ใ‘ใฉๆ‰‹ๅ…ƒใงdecisionใƒขใƒ‡ใƒซ็„ก้™ใซๅ›žใ›ใ‚‹ใจใ„ใ†ๅฎ‰ๅฟƒๆ„Ÿใฏ่จˆใ‚Š็Ÿฅใ‚Œใชใ„ใ€‚ไฝ•ใ‚ˆใ‚Šๅฐๅž‹ใชใฎใŒ่‰ฏใ„ใ€‚ใ‚ใ‚ŠใŒใจใ†liquidaiใ€ใƒญใƒผใ‚ซใƒซLLMใฏๆฒผใฎใƒ‹ใ‚ชใ‚คใŒใ™ใ‚‹ใชw
1
1
3
447
Tianshu Yu retweeted
Replying to @MiaAI_lab
The LFM3-VL-2B is the best model for its size for video understanding, quite usable compared to all other models of that size for checking security camera footage for example.
1
75
ๅฏฆ้š›้ซ”้ฉ—่ตทไพ† LFM3-VL-2B ๆฏ” LFM2.5-VL-3B ๅŽฒๅฎณๅคชๅคšไบ†ๅง๏ผ
1
2
311
LFM3-VL-2B vs Gemma 4 E2B on 50 real scanned receipts. LFM3 got 41 of 50 receipts fully right. Gemma got 28. 50 receipts from SROIE, same prompt, JSON out, one 5060 Ti 16 GB. - LFM3-VL-2B: 95% of fields right, 41/50 receipts perfect, 46 s - Gemma 4 E2B: 87% of fields right, 28/50 receipts perfect, 62 s Across both tests: Gemma reads messy chat-style text a bit better (95% vs 90%). LFM3 reads receipts better, and was faster both times. @liquidai @googlegemma
Introducing ๐™ป๐™ต๐™ผ๐Ÿน: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence. > 6 model instances, across text, audio, vision, and omni modalities, generative and decision models. > ๐™ป๐™ต๐™ผ๐Ÿน-๐Ÿท.๐Ÿฝ๐™ฑ: text, 256k context window. > ๐™ป๐™ต๐™ผ๐Ÿน-๐š…๐™ป-๐Ÿธ๐™ฑ: vision-language model with video-input support for spatial understanding > ๐™ป๐™ต๐™ผ๐Ÿน-๐™ฐ๐šž๐š๐š’๐š˜-๐Ÿธ๐™ฑ: tool calls, voice chat, reasoning in 18 languages. > ๐™ป๐™ต๐™ผ๐Ÿน-๐™พ๐š–๐š—๐š’-๐Ÿธ๐™ฑ: an omni model combines vision, audio, and language. > ๐š๐Ÿท-๐Ÿธ๐™ฑ and ๐š๐Ÿท-๐™พ๐š–๐š—๐š’-๐Ÿธ๐™ฑ: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input. > fine-tune on 1 GPU. > a new tri-hybrid architecture > architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs. > ๐™ป๐™ต๐™ผ๐Ÿน-๐š…๐™ป-๐Ÿธ๐™ฑ is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters. > ๐™ป๐™ต๐™ผ๐Ÿน-๐Ÿท.๐Ÿฝ๐™ฑ is #1 on the Pareto frontier of device intelligence ๐š™๐š’๐š™๐šŽ๐š๐š๐šŽ.๐š•๐š’๐šš๐šž๐š’๐š.๐šŠ๐š’ Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device. 1/ ๐Ÿงต
2
9
1
67
2,264
Tianshu Yu retweeted
pointed lfm3-vl-2b at the whole san fernando valley overnight. 16,692 aerial tiles, one rtx 3060, 5.5 hours. it found 62,873 pools and drew a box around each one. not a crazy idea, just โ€œfind the poolsโ€. it misses about 1 in 5 and boxes a roof now and then. but itโ€™s a 2B model with zero finetune. imagine what you can do with this. palantir at home lmao
Replying to @DogukanUrker
2/ LFM3-VL-2B 64 slots x 16K. ~143 tok/s on one request, ~583 tok/s total with all 64 describing images at once. 11.5GB VRAM. config: llama-server -m LFM3-VL-2B-Q8_0.gguf --mmproj mmproj-LFM3-VL-2B-F16.gguf -ngl 99 -c 1048576 -fa on --jinja -np 64 --cache-type-k q8_0 --cache-type-v q8_0 -b 2048 -ub 1024 --temp 0.1 --top-p 1.0 --top-k 50 --min-p 0.0 --repeat-penalty 1.1 --reasoning on --no-reasoning-preserve
14
28
7
450
17,062
Tianshu Yu retweeted
Introducing ๐™ป๐™ต๐™ผ๐Ÿน: our new generation of Liquid Foundation Models. With a new tri-hybrid architecture and new multimodal encoders, the LFM3 family is the ideal model suite for specialized intelligence. > 6 model instances, across text, audio, vision, and omni modalities, generative and decision models. > ๐™ป๐™ต๐™ผ๐Ÿน-๐Ÿท.๐Ÿฝ๐™ฑ: text, 256k context window. > ๐™ป๐™ต๐™ผ๐Ÿน-๐š…๐™ป-๐Ÿธ๐™ฑ: vision-language model with video-input support for spatial understanding > ๐™ป๐™ต๐™ผ๐Ÿน-๐™ฐ๐šž๐š๐š’๐š˜-๐Ÿธ๐™ฑ: tool calls, voice chat, reasoning in 18 languages. > ๐™ป๐™ต๐™ผ๐Ÿน-๐™พ๐š–๐š—๐š’-๐Ÿธ๐™ฑ: an omni model combines vision, audio, and language. > ๐š๐Ÿท-๐Ÿธ๐™ฑ and ๐š๐Ÿท-๐™พ๐š–๐š—๐š’-๐Ÿธ๐™ฑ: decision models with typed outputs and calibrated probabilities in a single pass. Omni adds audio input. > fine-tune on 1 GPU. > a new tri-hybrid architecture > architecture optimized for 14 processor classes across GPUs, CPUs, and NPUs. > ๐™ป๐™ต๐™ผ๐Ÿน-๐š…๐™ป-๐Ÿธ๐™ฑ is a beast. It performs competitively with Gemma 4 12B (6x larger), and Gemini 3.5 Flash Lite, and GPT-5.6 Luna, with 1.9B parameters. > ๐™ป๐™ต๐™ผ๐Ÿน-๐Ÿท.๐Ÿฝ๐™ฑ is #1 on the Pareto frontier of device intelligence ๐š™๐š’๐š™๐šŽ๐š๐š๐šŽ.๐š•๐š’๐šš๐šž๐š’๐š.๐šŠ๐š’ Every modality for specialized intelligence. Pick the one that fits your use case, customize it, and run it on-device. 1/ ๐Ÿงต
540
1,257
738
9,048
2,399,769