You don't like the Spark UI? We heard you! We're releasing Delight - a free & cross-platform monitoring dashboard for #ApacheSpark. datamechanics.co/blog-post/d… Backed by an #opensource agent, It works on top of any Spark platform - Databricks, EMR, Dataproc, CDH/HDP, ...

Our Data + AI summit talk on Running #Spark on #Kubernetes reliably on #spot instances is now on Youtube: youtube.com/watch?v=te9PaxKI… 1) Spark driver => On-Demand, Executors => spot. 2) Avoid 80% of spot kills by optimizing spot node selection 3) Handle executor kills gracefully

Delight, our free monitoring dashboard for Apache Spark, just got a big upgrade: spot.io/blog/a-new-look-for-… Delight gives you access to the Spark UI (we run the Spark History Server for you) and key system metrics (CPU, Memory, I/O) to help you understand the performance of Spark.

Data + AI Summit is a wrap! If you couldn't attend our session, we just released the same content as a blog post - How to run Spark on Kubernetes reliably on spot instances :) ! spot.io/blog/how-to-run-spar…

Ocean for Apache Spark, our fully managed, continuously optimized, Spark-on-Kubernetes service, is now available on Google Cloud. It's deployed on a GKE cluster in your GCP account. Learn more: spot.io/blog/ocean-for-apach… Congrats to the team! #AWS ✅ #GCP ✅ #Azure: coming next!

In this tutorial we show you how to run Jupyter Notebooks on our serverless Spark-on-k8s offering, Ocean for Apache Spark: spot.io/blog/tutorial-how-to… PS: In addition to Python notebooks, Scala notebook are now in private preview - reach out if you're interested!

Ocean for Apache Spark is now officially GA on AWS: spot.io/blog/ocean-for-apach… Main benefits of running Spark on k8s with us: intuitive UI, powerful automations enabling big savings, integrations with jupyter & airflow, ... Very proud of our team! GCP & Azure are coming next :)!

This step-by-step tutorial shows you how to schedule Spark pipelines with Airflow on our serverless, Spark-on-Kubernetes service, Ocean for Apache Spark: spot.io/blog/orchestrate-spa… We'll be doing a live demo of this in our webinar next week, sign up at: get.spot.io/spot-apachespark…

Since the Spark 3.2 release, the S3 Magic Committer is a lot more easy-to-use, performant, and stable. It enabled a 60% performance boost in our tests! Learn more in this technical blog post about #ApacheSpark performance tuning on #AWS #S3 spot.io/blog/improve-apache-…

Join us at our first webinar on how to get started with Apache Spark on Kubernetes! In this technical talk we'll show the differences between running Spark on k8s vs other architectures. We'll also share some of the secret sauce behind Ocean Spark! get.spot.io/get-started-with…

6 months after our acquisition by @spot_by_netapp, we're launching the joint @DataMechanics_ + Spot product called Ocean for Apache Spark. We make Spark on k8s more developer-friendly and cost-effective. Proud of our global team who made this possible! spot.io/blog/ocean-for-apach…

Step-by-step tutorial to help you run R workloads at scale with #ApacheSpark (SparklyR) on a #Kubernetes cluster. This was graciously contributed by one of our customers at @DataMechanics_ who's happy about the Docker-based dev workflow of Spark-on-k8s! datamechanics.co/blog-post/t…