A truly open-source LLM is out! OLMo by @ai2_allennlp is another step towards democratizing LLMs. With its 1B and 7B already hitting huggingface and 65B under training, there is a lot to look forward to! Here are some cool things about OLMo:
- Training and modelling code
- 500+ Model checkpoints
- Training data (Dolma) and analysis (WIBMD)
- Evaluation code (using Catwalk)
ALL OPEN-SOURCE!
Stay-tuned with @ai2_allennlp since soon they will be releasing:
- OLMo 65B
- Adapted OLMo with instruction-tuning and RLHF
- Training logs, ablation and Weights and Biases runs
Technical report: arxiv.org/pdf/2402.00838.pdf
Won't be able to attend ACL in Toronto since my visa application has been refused due to 'not a good enough reason to visit Canada'. 😐
Are biases unique to specific language representations based on the history & culture of the region they're spoken in? Check out our #NAACL2022 paper on “Socially Aware Bias Measurements for Hindi Language Representations“ (arxiv.org/pdf/2110.07871.pdf) with @kaiwei_chang @sunipa17