AI model built by the community, for everyone in this world Part of the Linux Foundation, Apache 2 licensed An RNN scaled to 14B params with GPT-level of perf
World
Joined November 2023
- Tweets70
- Following3
- Followers2.9K
- Likes62
Pinned Tweet
#RWKV is One Dev's Journey to Dethrone Transformers
The largest RNN ever (up to 14B). Parallelizable. Fast inference & training. Quantizable. Low vram usage.
3+ years of hard work
wiki.rwkv.com
Created by @BlinkDL_AI
Computation sponsored by @StabilityAI @AiEleuther
Overall we are excited by the progress we look forward to
- the step jump in RWKV7-based attention
- and the upcoming converted 70B models
Once properly tuned, and trained, it will serve as a full drop-in replacement for the vast majority of AI workloads today 🪿
Wrapping up: #RWKV was created by @BlinkDL_AI
as a project at @AIEleuther
- and is now being hosted by @LFAIDataFdn
The RWKV wiki can be found at: wiki.rwkv.com Our discord can be found at: discord.gg/bDSBUMeFpc
Give the models a try, drop by our discord
RWKV retweeted
🚀 RWKV.cpp AI system - is being deployed to 0.5 billion installs globally
Making it one of the world's most deployed, truly open-source (apache2) AI solutions out there
As it ships with every Windows 11 system today!!
Our group's code install count, went from ~100k -> 0.5B
Energy cost - Is critical for a device that aims for long battery life!
Variety of language support - is critical for shipping a product to the whole world (beyond English or Chinese)
We will eagerly keep watch on this development
blog.rwkv.com/p/rwkvcpp-ship…
RWKV.cpp project can be found here:
github.com/RWKV/rwkv.cpp
Our RWKV wiki can be found at:
wiki.rwkv.com
Our discord can be found at: discord.gg/bDSBUMeFpc
Give the models a try, drop by our discord, and have fun!
The RWKV v6 Finch lines of models are here
Scaling from 1.6B all the way to 14B
Pushing the boundary for an Attention-free transformer, and Multi-lingual models.
Cleanly licensedm Apache 2, under
@linuxfoundation
Find out more from the writeup here: blog.rwkv.com/p/rwkv-v6-finc…
As previously covered, this model is an upcycled model from our v5 - with an additional 1.4T tokens trained, for the conversion process.
The RWKV v6 Finch paper covering the architecture can be found here: arxiv.org/abs/2404.05892
Wrapping up: #RWKV was created by @BlinkDL_AI as a project at @AIEleuther - and is now being hosted by @LFAIDataFdn
The RWKV wiki can be found at:
wiki.rwkv.com
Our discord can be found at: discord.gg/bDSBUMeFpc
Give the model a try, drop by our discord
RWKV retweeted
Is AI's insatiable appetite for compute holding back innovation?
I recently talked with Eugene Cheah about @RWKV_AI. Eugene and the open-source RWKV community are working to democratize AI by addressing some of the key limitations of transformers.
Wrapping up:
#RWKV was originally created by @BlinkDL_AI as a project at @AIEleuther ; and is now being hosted by
@LFAIDataFdn
Compute for this training, was sponsored by @recursal_AI
You can find the latest EagleX model on their cloud platform here:
recursal.ai
The RWKV community wiki can be found at: wiki.rwkv.com
Our discord can be found at:
discord.gg/bDSBUMeFpc
Give the model a try, drop by our discord, and provide us feedback on how we can improve the model for the community.