RWKV (pronounced RwaKuv) is an RNN with GPT-level LLM performance, and can also be directly trained like a GPT transformer (parallelizable). We are at RWKV v6.
So it's combining the best of RNN and transformer - great performance, fast inference, fast training, saves VRAM, "infinite" ctxlen, and free text embedding. Moreover it's 100% attention-free, and a LFAI project.
Projects
RWKV-LM
Training RWKV v4 v5 v6
RWKV-Runner
RWKV GUI with one-click install and API for v4 v5 v6
RWKV pip package
Official RWKV pip package for v4 v5 v6
RWKV-server
Fast WebGPU inference (nVidia/AMD/Intel), int4/int8/fp16 for v4 v5 v6
RWKV.cpp
Fast CPU/cuBLAS/CLBlast inference, int4/int8/fp16/fp32 for v4 v5
nanoRWKV
Simple training, for any GPU / CPU
Misc
RWKV raw weights
All latest RWKV weights
RWKV weights
HuggingFace-compatible RWKV weights
RWKV wiki
Community wiki (with guide and FAQ)
