Eric Hofesmann (@ehofesmann) header
Eric Hofesmann (@ehofesmann) profile picture

Machine Learning + Computer Vision engineer @voxel51 CSE M.S. @UMich Love me some cooking šŸ², games šŸŽ², and dogs 🐶

Ann Arbor, Michigan
Joined September 2020
Photo by Eric Hofesmann (@ehofesmann)Photo by Eric Hofesmann (@ehofesmann)Photo by Eric Hofesmann (@ehofesmann)
Eric Hofesmann retweeted
In Physical AI, model architecture isn’t your bottleneck anymore. Your data flywheel is. šŸ¤– Watch the full clip with Daniel Gural from the Nov 11 Toronto Physical AI Workshop to see how to build a faster, leaner data flywheel for robotics and Physical AI - youtube.com/watch?v=jTeK1ssx… Leading robotics and autonomous vehicle teams are operating at the exabyte scale—combining synced cameras, LiDAR, radar, and joint telemetry. You can’t watch it all, and you can’t afford to annotate and train on repetitive footage (like a car pulling out of a garage every morning). The teams winning in Physical AI are the ones isolating the "golden subset" of edge cases and tightening their iteration loops: ⚔ Speed at scale: By pairing Voxel51 with NVIDIA Cosmos, teams across VW, Porsche, and Audi (CARIAD) cut their edge-case testing and retraining loop from weeks down to 8 hours. šŸ” Better diagnostics: Inspecting synchronized multimodal data (like MCAP recordings) lets engineers catch sensor bloat, spot overheating motors, and flag anomalies before the next training run. #PhysicalAI #Robotics #ComputerVision #MachineLearning #AutonomousVehicles #DataCentricAI #MLOps #Voxel51
1
2
5
143
Eric Hofesmann retweeted
240 human-corrected rollouts took a robot policy from 28% to 88% success in one manufacturing study. Successful demos can miss the states a policy reaches when it goes wrong. Recovery examples fill that gap. Learn more about how to curate a useful training set: link.voxel51.com/robot-polic… Try Voxel51 on your own data: link.voxel51.com/robot-polic…
1
2
4
1,323
Eric Hofesmann retweeted
most robot training runs are wasted on episodes that teach the policy nothing. the fix isn't a bigger model. it's finding the failures that matter and training on those. that's what we built. it's live on product hunt today and we need your help climbing šŸ‘‡ producthunt.com/products/vox…
1
3
7
319
Eric Hofesmann retweeted
Today at @NVIDIAGTC Berlin, @Porsche, @VW Group brands, and @cariad_tech announced that they are jointly building a shared AV data flywheel, with Porsche leading the initial implementation and ecosystem integration together with @Voxel51, @NVIDIA, and @Microsoft: link.voxel51.com/blfs3v
1
9
13
1,066
Eric Hofesmann retweeted
wassup big homie, i see you
2
3
14
488
Eric Hofesmann retweeted
@Voxel51 and @nebiusai are collaborating to help physical AI teams build the data and compute foundation they need to move faster. Join us at our upcoming workshops to run Voxel51’s data workflows on Nebius AI Cloud. Workshops in Silicon Valley, San Francisco, and more: voxel51.com/events?format=in…
1
4
1
10
784
Eric Hofesmann retweeted
The robotics data ecosystem keeps getting stronger 🦾 @Voxel51 now reads LeRobot datasets natively, with episode visualization, embedding exploration and semantic search across embodiments šŸ¤—
you balanced your robot dataset by embodiment. the embeddings say you balanced it by recording session i pulled 10 episodes from each of the 50 robot types in lerobot's community dataset and embedded every clip with qwen3-vl 1. 495 of 497 clips have their nearest neighbor in the same session 2. 292 score below 0.1 uniqueness 3. the two "most unique" clips are single-frame recordings, 0.02 seconds long filter the session out and the cross-embodiment signal shows up: a cloth fold on one rig lands next to so100 arms folding cloth i indexed all 497 episodes in fiftyone so you can search by text, cut the near-duplicates, and export a curated lerobot v3 subset one prompt to an agent with fiftyone skills built the whole thing start here, read the full blog: huggingface.co/blog/harpreet… read the dataset card: huggingface.co/datasets/Voxe… @LeRobotHF @Alibaba_Qwen
3
6
70
5,206
Eric Hofesmann retweeted
i've wanted fiftyone to load lerobot datasets for about 18 months it shipped this week so i did the only reasonable thing and loaded 11 of them. franka arms, a 10 camera flexiv rig, a unitree g1 doing chores in real homes, dexterous hands with fingertip tactile, a 27,500 episode bimanual sim one sample is one episode. every camera, the state vector, and the action scrub together on one timeline. that's the part i was waiting for every dataset has a live space you can open in your browser. check it out here: voxel51.com/blog/lerobot-v3-…
1
4
1
15
1,097
Eric Hofesmann retweeted
robot dexterity training data has a hard ceiling...someone has to teleoperate a physical robot apple removed the robot entirely people wore a vision pro and did everyday tasks. laundry, cards, shoelaces arkit tracked all 68 body and finger joints at 30 hz the whole time, on device, at recording time no annotators, no post-hoc pose estimation the result is a dataset called egodex. 338k episodes, 194 tasks, 829 hours, published at ICLR 2026 apple calls it the largest and most diverse dexterous manipulation dataset to date. it also shows what the vision pro quietly is...a hand-tracking data factory i parsed the held-out test set, 3,243 episodes across 111 tasks, into fiftyone as multimodal MCAP: egocentric video, a 3d animated skeleton, and the 2d pose overlay projected onto the frames, all synced at 30 hz. filter by task, task type, objects, or language description scrub an episode in the browser, no install: huggingface.co/spaces/harpre… dataset card: huggingface.co/datasets/Voxe…
1
2
13
1,190
Eric Hofesmann retweeted
most driving datasets stop annotating at 80 meters. at highway speed that's about 2 seconds of warning aimotive annotates out to 200 meters, and roughly a quarter of its 425k boxes sit beyond 75 m lidar, four cameras, and two radars in a 360 layout, so every object is covered by at least two sensor types 176 scenes across california, austria, and hungary in daylight, night, and rain the training labels were written by a non-causal tracker that gets to watch the whole 15-second clip before deciding where a car was and validation labels were written by humans. your model trains on hindsight and gets graded by people i parsed all 176 episodes into MCAP for fiftyone four camera streams with projected boxes, the point cloud with colored 3d cubes, raw radar returns, and gps on one synced timeline. filter by condition to pull up only night or only rain episodes start here, read the dataset card: huggingface.co/datasets/Voxe… live demo in a hugging face space: huggingface.co/spaces/harpre…
2
8
1
61
60,922
Eric Hofesmann retweeted
a policy can watch ten thousand videos of people opening drawers and still not know how hard to pull video records what the hand did. it never records what the hand felt hoi! from ETH zurich records a handheld gripper instrumented with a 6-axis force-torque sensor at 100 hz, gelsight tactile images on both fingertips, and cameras on the human's glasses and the gripper itself, all registered to a millimeter-accurate laser scan of the room the same drawers, fridges, and cabinet doors operated with bare human hand, UMI gripper, and instrumented gripper. allowing researchers to study what actually transfers when the body changes i packaged 88 episodes from the 3,048-sequence CVPR 2026 release as MCAP for fiftyone you can scrub the force plot next to the tactile images next to both camera views on one timeline, with saved views per embodiment. no install, explore it in the browser: huggingface.co/spaces/harpre… full dataset: huggingface.co/datasets/Voxe…
1
3
18
1,107
Eric Hofesmann retweeted
most lidar datasets pay annotators to decide which points are moving. one of the sensors in this dataset labels itself highwayscene points three lidars with three different sensing principles at the same stretch of german highway at the same time: a spinning ouster OS0, a solid-state blickfeld, and an aeva FMCW unit the FMCW one measures radial velocity per point. all ~92k points per frame arrive with their own speed, so "dynamic" is just physics: anything over 1 m/s. the other two sensors need hand-drawn lane volumes to earn the same label built by esslingen university for their ITSC 2026 cross-sensor background subtraction benchmark. same traffic, three completely different point clouds which is exactly the test your method skips when it's tuned on one sensor i converted the protobuf records to MCAP so all 30 episodes load in fiftyone's multimodal viewer with one line, three point clouds scrubbing on a shared timeline checkout the dataset here: huggingface.co/datasets/Voxe… it's running as a live space too, nothing to install: huggingface.co/spaces/harpre…
4
13
983
Eric Hofesmann retweeted
you have budget for a few hundred annotations and a pool of thousands of unlabeled images. pick randomly and you relabel what you already understand while the classes you need most stay uncovered on a real 1,754-image drone inspection dataset: random sampling finds the hardest class 12.9% of the time. targeted search + embedding-based prioritization finds it 100%. same labeling budget live workshop, the full loop: 1. embed the pool once 2. compress duplicates 3. search with a few labeled examples 4. prioritize what's left 5. fine-tune, then correct instead of relabel aug 20 Ā· free Ā· hands-on in fiftyone register: voxel51.com/events/from-cold…
3
29
1,495
Eric Hofesmann retweeted
a robot can navigate a hallway without hitting anyone and still make every person in it uncomfortable. collision-free and socially aware are two completely different problems NavWareSet records both. seven social navigation scenarios (frontal approach, blind corner, following, perpendicular crossing), each run twice under matched conditions: once with socially compliant behavior, once without. same room, same people, same starting positions. the only variable is whether the robot navigates like it knows humans have personal space robot onboard lidar and camera plus an overhead ground truth station tracking every pedestrian in 3D across the full episode loaded as native mcap in fiftyone. scrub the robot's camera, both lidar streams, and the annotated pedestrian trajectories on one synced timeline. filter by scenario and behavior to compare compliant vs non-compliant side by side start here, read the dataset card: huggingface.co/datasets/Voxe…
5
24
1,505
Eric Hofesmann retweeted
1 in 3 teams pay to annotate data that never makes it to production. That's not a labeling problem. That's a strategy problem - hubs.ly/Q04pvP2r0 FiftyOne Annotation treats annotation as a data flywheel — not a one-time chore. * Label the right data first. Smart Data Selection surfaces the rare, high-signal samples that actually move model performance — teams ship with 60–80% less annotated data. * Agentic labeling. VLMs handle the first pass, so humans review and correct instead of drawing every box from scratch. * 2D, 3D & video in one place. Bounding boxes, SAM2 click-to-segment, lidar cuboids, temporal events — all native. * Intelligent Review. Embeddings + model comparison catch systematic label errors before they poison training. * Close the loop. Model eval failures feed straight back into the next annotation round. Every iteration compounds. The magic isn't any single feature — it's that curation, annotation, and evaluation finally live in ONE platform. No handoffs. No lost context. Just faster models, less waste. Stop labeling everything. Start labeling what matters. #computervision #ai #artificialintelligence #machinevision #machinelearning #datascience #physicalai #mcp #agents
1
5
6
238
Eric Hofesmann retweeted
Builders gonna build! Eric Hofesmann from the @Voxel51 rolled out a new #FiftyOne plugin last week for interactive visualization of the hyperspectral imagery including the ability to vary the transfer function dynamically. Check out the plugin here with instructions on how to install it: github.com/ehofesmann/envi-s…
2
9
395
Eric Hofesmann retweeted
PaliGemma2-Mix is now integrated into FiftyOne! You can use this model for: • Image captioningĀ (multiple detail levels) • Object detection • Semantic segmentationĀ (Not perfect, but good for initial exploration) • Optical character recognition (OCR) • Visual question answering • Zero-shot classification All with just a few lines of code! Check out the example notebook here: github.com/harpreetsahota204…
1
3
182
Eric Hofesmann retweeted
If you’re deploying models into the wild, this new open source technique for visualizing model certainty on unlabeled data at scale could change the game for you. ā¬‡ļø In our latest post, we dive into: āœ… A simple method for turning model outputs into actionable certainty scores āœ… How we used this technique across 190 ResNet18 models, each trained on a unique variation of theĀ CIFAR100Ā training set āœ… Why typical embeddings fall short—and what works better This is a great way to answer the question AI developers really care about: will our deployed model fail or not, and if so, on which data? Read the full breakdown:Ā voxel51.com/blog/visualizing… #ComputerVision #AI #VisualAI #MachineLearning #ModelEvaluation #VisualAI #OpenSource #FiftyOne
1
7
144
Eric Hofesmann retweeted
Selective Transparency and The Battle for Open Source What a great way to start a new week other than a new contributed article in VentureBeat!!! Focus: the battle for open source AI through the serious risk that selective transparency poses. venturebeat.com/ai/the-open-…
3
6
296
Eric Hofesmann retweeted
DeepSeek AI is on fire! Just yesterday they shipped Janus-Pro, a Unified Multimodal Model for multimodal understanding and text-to-image generation. The model comes in two model sizes: • Janus-Pro-1B, which is a little over 1.5 billion parameters • Janus-Pro-7B with 7 billion parameters. According to the technical report, the model can perform several vision-language tasks, such as : • Image descriptions • Landmark recognition • OCR Interestingly, the authors claim that this model excels in meme perception. Oh, wait a minute...I read that wrong in the report...that doesn't say MEME perception. It says MME perception. Well, anyway, I'm gonna see just how well this model does at meme perception, specifically for: • extracting captions from a meme • understanding a meme • generating captions for a meme And in the spirit of competition, I'll pit it head to head against another newly released model: moondream2 šŸ‹ vs šŸŒ• - Let's go šŸš€! Code and all you need šŸ‘‡šŸ¼ #deeplearning #artificialintelligence #computervision #genai
1
1
6
274