Gabriele Goletto retweeted
*NEW PAPER&MODEL*
HOI-DETR for detecting hands, object in-hand, &object-interacted-with (through a tool).
This is the foundation model you've been waiting for! One that works off-the-shelf, single-image, using strong detectors, generalises and is stable
ahmaddarkhalil.github.io/HOI…
🧵
Gabriele Goletto retweeted
After two fantastic years at @UCBerkeley I'm thrilled to share that I've joined @Microsoft in Zurich🇨🇭to pioneer the next generation of multimodal foundation models to drive agents 🤖 that can seamlessly interact across the digital and physical worlds 🌍
We are hiring! 🧵
Gabriele Goletto retweeted
🚀 As #CVPR2025 week kicks off, meet SANSA: Semantically AligNed Segment Anything 2
We turn SAM2 into a semantic few-shot segmenter:
🧠 Unlocks latent semantics in frozen SAM2
✏️ Supports any prompt: fast and scalable annotation
📦 No extra encoders
📎 github.com/ClaudiaCuttano/SA…
Gabriele Goletto retweeted
Now on ArXiv our @CVPR #CVPR2025 paper
Learning from Streaming Video with Orthogonal Gradients
Instead of shuffling clips, can we learn from videos fed sequentially, where you see a clip once, in order?
How to deal with the correlation of gradients over training?
1/3
Replying to @TengdaHan
Check out our CVPR 2025 paper: arxiv.org/abs/2504.01961.
Work with Dilara Gokay, Joseph Heyward, @ChuhanZhang5 , @DanielZoran_ , Viorica Pătrăucean, @joaocarreira , @dimadamen and Andrew Zisserman, @GoogleDeepMind
Gabriele Goletto retweeted
Image segmentation doesn’t have to be rocket science. 🚀
Why build a rocket engine full of bolted-on subsystems when one elegant unit does the job? 💡
That’s what we did for segmentation.
✅ Meet the Encoder-only Mask Transformer (EoMT): tue-mps.github.io/eomt (CVPR 2025)
(1/6)
Gabriele Goletto retweeted
🔥 Our paper SAMWISE: Infusing Wisdom in SAM2 for Text-Driven Video Segmentation is accepted at #CVPR2025! 🎉
We make #SegmentAnything wiser, enabling it to understand text prompts—training only 4.9M parameters! 🧠
💻 Code, models & demo: github.com/ClaudiaCuttano/SA…
Why SAMWISE?👇
Gabriele Goletto retweeted
🛑📢
HD-EPIC: A Highly-Detailed Egocentric Video Dataset
hd-epic.github.io
arxiv.org/abs/2502.04144
New collected videos
263 annotations/min: recipe, nutrition, actions, sounds, 3D object movement &fixture associations, masks.
26K VQA benchmark to challenge current VLMs
1/N
Gabriele Goletto retweeted
📢 Our @ACCVConf Oral
It’s Just Another Day: Unique Video Captioning by Discriminitive Prompting
is now on ArXiv
tobyperrett.github.io/its-ju…
For challenging ego &timeloop movies, uniquely caption ev clip, including those near identical ones, w/out re-training captioning model
1/N
Gabriele Goletto retweeted
Gabriele Goletto retweeted
Love timm, but need to do lower-level vision tasks? Meet IMM, a collection of Image Matching Models unified by a simple, easy to use API. You can use any of 30 models (LightGlue, LoFTR, RoMa, SIFT) just by cloning the repo and changing a single parameter! github.com/gmberton/image-ma…
Gabriele Goletto retweeted
This work aims to detect and identify activity-centric zones in real-world conditions and leverage their domain-agnostic representations to enhance the generalization of first-person action recognition models.
Paper: Egocentric zone-aware action recognition across environments
Link: arxiv.org/abs/2409.14205
Project: gabrielegoletto.github.io/Eg…
#AI #AI美女 #LLMs #deeplearning #machinelearning #3D #actionrecognition
Gabriele Goletto retweeted
AMEGO - a representation of long videos. AMEGO breaks the video into Hand-Object Interaction (HOI) tracklets, and location segments. This forms a semantic-free memory of the video. AMEGO is built in an online fashion, eliminating the need to reprocess past frames.
Paper: AMEGO: Active Memory from long EGOcentric videos
Link: arxiv.org/abs/2409.10917
Project: gabrielegoletto.github.io/AM…
#AI #AI美女 #LLMs #deeplearning #machinelearning #3D
Gabriele Goletto retweeted
📢 [New Preprint]@eccvconf #ECCV2024 paper
AMEGO: Active Memory from long EGOcentric videos
gabrielegoletto.github.io/AM…
Semantic-free representation of all interacting objects and locations in a long egocentric video.
w/ 20K VQA benchmark that uses object crops to query interactions.
Gabriele Goletto retweeted
Our retrieval + matching pipeline for astronaut photo localization is out! Manually localized photos taken by @Space_Station astronauts are used for disaster management and research - we automate this process with EarthLoc (CVPR24) and EarthMatch (CVPRW24) earthloc-and-earthmatch.gith…
Gabriele Goletto retweeted
Waiting @eccvconf reviews - need a distraction?
Check our accepted #IJCV paper
"An Outlook into the Future of Egocentric Vision".
Read about Stanley, our EgoDesigner using #EgoAI to design the set of a movie in 2030, or Marco the factory worker w/ #EgoAI
sid2697.github.io/futureofeg…
Gabriele Goletto retweeted
Enabling VLMs to understand hand-object interactions in egocentric vision! We use EPIC-Kitchens & @ego4_d to train VLM4HOI.
Check the paper to know how we did it arxiv.org/abs/2404.09933
Data: github.com/Sid2697/HOI-Ref/b…
Code: github.com/Sid2697/HOI-Ref
Webpage: sid2697.github.io
🔔Can VLMs spatially refer objects in Ego?
Can VLMs understand interactions? Which hand is holding an object and what's the object in the left hand?
We show current VLMs struggle in interactions and release new data & models for HOI-Ref in Ego
sid2697.github.io/hoi-ref/
On ArXiv🧵
Gabriele Goletto retweeted
A revised version of our paper "An Outlook into the Future of Egocentric Vision" is now available.
This includes the latest peer-reviewed works after our initial submission.
*New* sections on Ego-Language models added and novel insights included
arxiv.org/abs/2308.07123
📢 "An Outlook into the Future of Egocentric Vision"
44 pages + 385 references survey now available on @openreviewnet
We invite comments/suggestions/corrections from researchers for 30days.
Major contributions will be acknowledged
[instructions in 🧵]
openreview.net/forum?id=V397…
1/4
Gabriele Goletto retweeted
Applications now open- 2nd Summer of Research @BristolUni #MachineLearning and #ComputerVision (MaVi) group.
R u PhD student with overlapping interests to us?
U can visit for 3mnths this summer!
Apply - DL 19/1/24
uob-mavi.github.io/Summer@Ma…
Watch 2023 cohort: youtu.be/p12V1z2Te0Y
Gabriele Goletto retweeted
📢"An Outlook into the Future of Egocentric Vision" 44 pages + 385 references survey now available on
@openreviewnet
We invite comments/suggestions/corrections from researchers for 30days. Major contributions will be acknowledged [instructions in 🧵]
openreview.net/forum?id=V397…
1/4

























