For years, I was an open source skeptic. I didn’t think open models would ever close the gap with frontier labs and I was wrong. I’ve moved all my research to open source models. Over the next 3-4 months, I’m putting around $1M into my own mini data centre to train and specialize them. The amount of tokens I have spent on frontier labs have been astronomical for the kind of results I was looking for and it has cost me hundreds of thousands of dollars in short period of time and it’s still restricted in speed and quality which isn’t enough anymore for impactful outcomes. A model doesn't need to be great at everything. A team of specialized models, each trained deeply in its own field and working together, can beat a single generalist. I believe that approach can outperform today's frontier models like Fable and Astra by a wide margin. With the current advancement in AI I believe the GPU prices will skyrocket even further and soon it’ll be too expensive to be even able to run open source models even for people who can afford it today. I’m excited for what’s coming. I’m working a lot in silence like a maniac, it’s advancing well, I’m sure the results will follow very soon.