Todd Mostak

Today, I'm excited to share that the full HeavyAI platform is now open source under the Apache 2.0 license: our GPU-accelerated database, HeavyDB, along with our rendering engine and visualization frontend. github.com/heavyai/heavydb Nearly 15 years ago, I started building what became HeavyAI to answer questions beyond the reach of existing tools. Working with Tarek Masoud at Harvard Kennedy School, and collaborating closely with @HarvardCGA, we tried to analyze hundreds of millions of geolocated tweets to better understand the Arab Spring in Egypt. I'd find that my queries on CPU would take hours or days, only in many cases to run out of memory, and so using the Nvidia 560 Ti under my desk, I taught myself CUDA and started to explore GPU-acceleration as a way allow me to explore the data interactively and at scale. I was fortunate to continue that work with @SamuelMadden in @MIT_CSAIL's database lab. It soon became clear that many others shared the same frustration: they had the data, but couldn't interactively explore it. That led me to found MapD (Massively Parallel Database) in 2013. The name changed over the years, first to OmniSci and then HeavyAI, but the goal stayed the same: help people get answers from their data at the "speed of curiosity." We built a SQL engine that ran on NVIDIA GPUs and CPUs, powered by an LLVM-based query compiler, and a rendering engine that made massive datasets explorable through interactive visualization, particularly for geospatial analytics. None of this would have happened without the team who built it, the customers who pushed us, and the partners and investors who believed in us. Special thanks goes to @JensenHuang and the @nvidia team, who saw the potential of accelerated data processing early on, and to @NEA, Tiger Global, @VanedgeCapital, @GVteam, @VZVentures, @InQTeI, and @DcodeCapital. And to our customers across telco, financial services, energy, utilities, and the public sector: thank you for trusting us with your hardest data problems. Now that the HeavyAI team and I are at NVIDIA, we're more convinced than ever that accelerated computing will play a central role in the future of data processing. We're working with partners across the ecosystem to help make that happen. We're already seeing GPU acceleration become part of mainstream analytics platforms alongside their existing CPU engines. That said, I still think specialized platforms like HeavyAI have an important role to play, for example where interactive visualization and geospatial analytics need to work together at scale. Try the platform, explore the code, and tell us what you build with it. I started this project to answer questions I couldn't easily explore with existing tools. With the full platform now open source, I'm excited to see what people build with it.

Truly awesome to see Jensen in his GTC keynote present @nvidia's joint partnership with @IBM to use cuDF to accelerate open source Velox and the Presto SQL engine in watsonx, showcasing how @Nestle achieves 5X faster queries for 83% less cost. Amazing collaboration between the Nvidia and IBM teams to make this a reality, and we're just getting started.

If it was already difficult to know what is real, we are quickly going to converge on a world where representation replaces reality. Generated with Sora 2 Pro.

Sora 2 Pro is really good. I asked it to recreate the scene of the 2010 Tahrir Square protests and it pulled together quite a convincing scene, complete with anti-Mubarak chants calling for the fall of the regime.

Sora 2 is just wild…

I've been diving deeper into the @Foursquare Places dataset that was released two weeks ago, and am finding it great not only for accessing individual POIs, but also for performing large-scale analysis of geographic trends. Here I'm using @heavy_ai to join the Places dataset to the NASA Gridded Population of the World (GPW) dataset, containing population estimates per square km for the entire world, using Uber H3 hexagonal spatial indexes as join keys. This allows for easy calculation and visualization of the number of POIs by category per capita. See below for two visualizations comparing the number of places of worship with the number of bars per 1,000 people. You can easily see the Bible belt light up for the former category, but why the high prevalence of both places of worship and drinking establishments in the Upper Midwest?

Was super excited this morning to see @Foursquare had open sourced a dataset over 100M POIs. I immediately it into @heavy_ai and am having a lot of fun exploring the data, it looks fantastic both in terms of coverage and accuracy.

#30DayMapChallenge Day 3: Polygons New @ORNL US buildings dataset, complete with addresses and height data. Here are building footprints in the Philadelphia area, colored by height in @heavy_ai.