Pinned Tweet
1/ Real2Sim and Sim2Real are well-known paradigms in robotics🤖. Is there another paradigm where sim and the real world are more tightly interlinked? We introduce Real-IS-Sim: a framework featuring an always-in-the-loop, correctable simulator for behavior cloning policies.
Grateful to @NVIDIA for featuring our work on their Tech Blog: Physically Embodied Gaussians, an interpretable real time world model for #robotics using Warp and #GaussianSplatting. developer.nvidia.com/blog/bu…
@krshnrana @nikoSuenderhauf @QUTRobotics
🚀 Embodied Gaussians is now opensource! 🔥
📂 GitHub: github.com/bdaiinstitute/emb…
✨ Key Features:
1️⃣ Gaussian Splatting (GSplat) + Physics (Warp) 💥
2️⃣ Parallelizable environments! ⚡
3️⃣ Render virtual cameras in parallel 📸
🔜 Coming Soon (hopefully):
🌀 Deformables!
5/ This project was completed during my internship at @rai_inst (formerly Boston Dynamics AI). Special thanks to my team—Lingfeng Sun, @krshnrana, Brandon May, Karl Schmeckpeper, Maria Vittoria Minniti, and Laura Herlant for their support during this project.
1
1
769
6/ Videos and paper are available at realissim.rai-inst.com/ and arxiv.org/abs/2504.03597
1
8
703
1/ What is a good representation of the physical world for a robot🤖? We believe it is one that is: initialized easily, 3D, physically constrained, correctable with new observations (in realtime), and capable of being forward simulated (faster than realtime).
4/ Particles collide with each other and are affected by external forces and kinematic constraints. The particles transport their linked Gaussians along with them. The system runs in realtime @ 30Hz with only 3 cameras. It works on both deformables and rigid objects.
1
9
901
5/ More videos and details are available at the project page (embodied-gaussians.github.io…) and in the paper (arxiv.org/abs/2406.10788).
A special thanks to @krshnrana, Feras Dayoub, @nikoSuenderhauf, and @QUTRobotics for their support during this project.

