The Ongoing Case For Open Source LLMs Custom LLMs, long context, and efficient inference Some folks believe that training open-source LLMs is a losing battle and a complete waste of time. They argue that the gap between closed models like GPT-4 and open models like Llama will widen and these open-source models may never catch up. Yes, closed models like Google's Gemini or Open AI's Gobi promise to be way more powerful than GPT-4, so what hope does open source have? To start with inference on GPT-4 is very expensive. These very large models may be performant but aren't cost-effective. At Abacus, we routinely use fine-tuned versions of LLama-2 and smaller models when we need to run 1M+ API calls a day for standard enterprise applications Q/A, summarization, and NLP at scale. GPT-4 would cost > $100K a day in these cases. Instruct-tuned LLMs can match the performance of GPT-4 for a specific task. Instruction tuning is a technique that aims to improve the capabilities and controllability of LLMs. It involves further training these models on a dataset consisting of (instruction, output) pairs in a supervised manner. This bridges the gap between the next-word prediction objective of LLMs and the users' objective of having LLMs adhere to human instructions. For example, we have instruct-tuned open-source models for tasks like Q/A, NER, and classification. These instruct-tuned models are better at generalizing the task to new data and can do so in a resource-efficient manner. Another shortcoming of currently available closed models like GPT-4 is that they have relatively short context lengths. 8K tokens are default and this means that you can't pass it large documents and ask it to extract the results from there. Luckily the open-source community has been busy solving practical problems like this. Earlier this week, the paper LongLoRA introduced an ultra-efficient fine-tuning approach to significantly extend the context windows of pre-trained LLMs. LongLoRA adopts LLaMA2 7B from 4k context to 100k, or LLaMA2 70B to 32k on a single 8x A100 machine, and basic implementation only takes 2 lines of code. Open-source LLMs have been the focus of the GPU-poor and constraining resources typically have magical effects - efficient, elegant, and simple innovations that solve the problem! Open-source LLMs have emerged as cheap and efficient alternatives for enterprise AI use cases and will continue to play an important role in the space. Some have argued that open-sourcing LLMs is dangerous and they may be misused by bad actors. There is no historical precedent for this argument. Traditionally, open-source technology has spurred innovation, transparency, and the creation of safe and robust systems. Linux, triumphed over Unix in the OS world, largely because it is open-source and has a huge developer community. Open source promotes collaboration, community oversight, rapid iteration, and benchmarking all essential for responsible AI development. Open-source developer communities tend to be great at detecting and plugging vulnerabilities. Disappointingly, big players like OpenAI (despite their name) and Google, haven't open-sourced a lot of their technology. Luckily for the open-source community, Meta has created accessible open-source LLMs. In spite of Meta open-sourcing the powerful 70B LLama-2., the doomsday scenarios outlined by the anti-open-source crowd haven't come true. Finally, multimodal LLMs (MLLM) are around the corner and if the GPU-rich won't outsource a MLLM, we can always enhance an existing open-source LLM and convert it into a multi-modal model. In summary, open-source LLMs play a role in the real-world application of AI and are crucial for the democratization of this technology, transparency, and AI alignment