Zhenzhen Weng retweeted
Just published in @ScienceAdvances, our work demonstrating the ability of AI and 3D computer vision to produce automated measurement of human interactions in video data from early child development research -- providing over 100x time savings compared to human annotation and enabling quantitative, big data studies. We use our method, HARMONI, to characterize longitudinal trends in infant and toddler interaction with caregivers, in over 500 hours of video data. Work led by @JenWeng4 together with co-PI @SandersMDMPH and @K_L_Humphreys, and with a great interdiscplinary team including Laura Bravo Sanchez, @bergelsonlab, @akanazawa, @StanfordCERC, and many others!
science.org/doi/10.1126/scia…
Zhenzhen Weng retweeted
1/3 Today, an anecdote shared by an invited speaker at #NeurIPS2024 left many Chinese scholars, myself included, feeling uncomfortable. As a community, I believe we should take a moment to reflect on why such remarks in public discourse can be offensive and harmful.
Zhenzhen Weng retweeted
Our lab at Stanford has postdoc openings! Candidates should have expertise and interests in one or multiple of: multimodal large language models, video understanding (including video-language models), AI for science / biology, or AI for surgery. Please send inquiries by email and see marvl.stanford.edu for more information.
Zhenzhen Weng retweeted
🤗First benchmark on multimodal judge’s feedback for text-to-image generation!!
🏃Come and pick up your personal advice and package to choose the best judge to fine-tune your diffusion model 👉 mj-bench.github.io
Paper: huggingface.co/papers/2407.0…
Code: github.com/MJ-Bench/MJ-Bench
🌟NEW Paper Alert 🌟
👩⚖️MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? (mj-bench.github.io/)
🧐Also wonder about the best judge model to provide feedback for your diffusion models?
We evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias.
Key findings:
👉1. While closed-source VLM judges typically perform better, smaller CLIP-based models offer better text-image alignment and image quality feedback due to extensive pre-training on text-vision corpus. Conversely, VLMs provide more accurate feedback on safety and generation bias, thanks to their stronger reasoning capabilities.
👉2. VLM judges can provide more accurate and stable feedback in natural language (e.g. Poor, Average, Good) than numerical scales.
Led by @ZRChen_AISafety, Yichao Du, Zichen Wen, @AiYiyangZ.
arxiv.org/pdf/2407.04842
Zhenzhen Weng retweeted
🌟NEW Paper Alert 🌟
👩⚖️MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation? (mj-bench.github.io/)
🧐Also wonder about the best judge model to provide feedback for your diffusion models?
We evaluate multimodal judges in providing feedback for image generation models across four key perspectives: alignment, safety, image quality, and bias.
Key findings:
👉1. While closed-source VLM judges typically perform better, smaller CLIP-based models offer better text-image alignment and image quality feedback due to extensive pre-training on text-vision corpus. Conversely, VLMs provide more accurate feedback on safety and generation bias, thanks to their stronger reasoning capabilities.
👉2. VLM judges can provide more accurate and stable feedback in natural language (e.g. Poor, Average, Good) than numerical scales.
Led by @ZRChen_AISafety, Yichao Du, Zichen Wen, @AiYiyangZ.
arxiv.org/pdf/2407.04842
Zhenzhen Weng retweeted
If you’re in Davos, we just started giving a tutorial on Gaussian Splatting at 3DV. With @GKopanas @Snosixtytwo @antoine_guedon
3dgstutorial.github.io/
3dvconf.github.io/2024/tutor…
Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis
dynamic3dgaussians.github.io…
We model the world as a set of 3D Gaussians that move & rotate over time. This extends Gaussian Splatting to dynamic scenes, with accurate novel-view synthesis and dense 3D trajectories.
Zhenzhen Weng retweeted
Thanks @_akhaliq for sharing our work! Letting LLM be an agent and long-form videos as an environment, and allowing LLM to interact with videos and decide where to look iteratively, we achieve SoTA zero-shot performance and show potential on processing extremely long videos!
Zhenzhen Weng retweeted
Are you hiring top AI talent?
Here is a list of Ph.D. students affiliated with @StanfordAILab who are on the industry and academic job markets this year! This list showcases diverse research areas and 41% of these graduates are URMs!
Check it out: ai.stanford.edu/blog/sail-gr…
Zhenzhen Weng retweeted
What are differences between image datasets? (e.g. ImageNet & ImageNetv2) Errors by one model vs. another? (e.g. CLIP & ResNet) Correct vs. incorrect predictions?
VisDiff can answer by describing differences in image sets w/ language. Work led by @Zhang_Yu_hui and @lisabdunlap!
[1/5] Introducing VisDiff - an #AI tool that describes differences in image sets with natural language.
VisDiff can summarize model failures, compare models, find nuanced dataset differences, discover what makes an image memorable, and so much more!
understanding-visual-dataset…
Zhenzhen Weng retweeted
Sneak peek in video GenAI projects we’re working on @AdobeResearch :
1/3 compositing videos in 3D with NeRF
#ProjectSceneChange
You’ll be able to put yourself in any scene including your artwork.
#AdobeMAX #CommunityxAdobe
Zhenzhen Weng retweeted
Check out Adobe's Project Scene Change
The interesting experimental #AI tech automatically composites an actor from one shot into the environment from another without the need for #rotoscoping or #cameratracking
cgchannel.com/2023/10/sneak-…
#compositing #VFX #motiongraphics
Zhenzhen Weng retweeted
Have videos of your tennis practice and wish you can put your own motion in 3D? 🎾 👟 🏋🏻
#CVPR2023 We present, NeMo, a 3D motion recovery method that is more accurate by leveraging information shared across multiple instances/repetitions!
👇🏻Resources in 🧵
Zhenzhen Weng retweeted
Check out our #CVPR2023 paper on 3D Human Keypoints Estimation From Point Clouds in the Wild Without Human Labels youtu.be/vXGPW2nDHZ4
Huge shout out to @JenWeng4 who interned in our team last summer and did all the work!
Zhenzhen Weng retweeted
(1/8)
Can you diagnose and rectify a #vision model using #language? Check our work in #ICLR2023!
Our analysis reveals when and how text embeddings can be used as a proxy for image embeddings to debug vision models.
Paper: arxiv.org/abs/2302.04269
Code: github.com/yuhui-zh15/drml
Zhenzhen Weng retweeted
Releasing 🌟nerfstudio🌟, a plug-and-play python library to easily create your own NeRFs!
@nerfstudioteam is a contributor friendly open-source repo with a realtime web viewer that makes it easy to make cool videos 📽️
github.com/nerfstudio-projec…
docs.nerf.studio #nerfstudio 1/
Zhenzhen Weng retweeted
Andrew Marmon and I rounded up all #CVPR2022 papers on NeRF/Neural Radiance Fields we could find in a new blog post here: dellaert.github.io/NeRF22/
Zhenzhen Weng retweeted
Very excited to present our paper @BMVCconf this Wednesday (day 3, session 3)!
Presenting FlowVOS: Weakly-Supervised Visual Warping for Detail-Preserving and Temporally Consistent Single-Shot Video Object Segmentation, a labor of love with @HNSurgeon and @syeung10. #BMVC2021
Zhenzhen Weng retweeted
Excited to teach CS 523 + talk w/ amazing folks like @syeung10 & Jen Weng & @HNSurgeon, @liu_xingtong & @MathiasUnberath, Tae Soo Kim & @GregGdhager, @RyutaroT92, @meenakshik93 & @mertrory, @AdrianDalca, @dc_moyer, @NanWu__, @aktanwani, @AndreEsteva!! #MedicalAI #AcademicTwitter
Enroll in a new spring qtr course: "Research Seminar in Computer Vision and Healthcare" (CS 523) by @julia_gong on Wednesdays, 11:30am-12:30pm PST. Class starts on 3/31, learn more here: web.stanford.edu/class/cs523…
Open to @Stanford & affiliates to audit. #AI #Healthcare #Stanford




























