About Me
I’m a Principal Applied Scientist working on multi-modal foundation models (video generation models in particular) at Amazon Artificial General Intelligence (AGI) org. In the past, I was with AWS AI Labs, and Meta Reality Labs where I worked on several initiatives on 3-D computer vision.
I received my Ph.D. in Computer Science from UCLA in 2019, where I worked with Prof. Stefano Soatto at the UCLA Vision Lab. My dissertation, Inertial-aided Visual Perception of Geometry and Semantics, explored how inertial measurements can improve visual understanding of geometry and semantic structure.
My research spans computer vision, robotics, and machine learning. I am particularly interested in building models and systems that solve real-world problems by combining information across sensors and modalities.
Our paper Geo-Supervised Visual Depth Prediction, which uses inertial measurements and gravity-induced shape priors to improve monocular depth prediction, received the Best Paper Award in Robot Vision at ICRA 2019, selected from 2,900 submissions.
I received my B.Eng. in Information and Communication Engineering from Zhejiang University in 2014. I was also a member of the Advanced Honor Class of Engineering Education at Chu Kochen Honors College, where I developed an enduring interest in mathematical modeling and interdisciplinary engineering.
My current CV is available here.
Awards & Distinctions
- Best Paper Award in Robot Vision, ICRA 2019.
- Meritorious Winner of Mathematical Contest in Modeling, 2013.
- National Scholarship, Ministry of Education, China.
Demo
Amazon Nova Video and Image Generation Models
Multi-Sensor Localization and Mapping
Additional Demo
- Visual-Inertial Navigation and Semantic Mapping, CVPR 2016 Demo: [video] [poster]
- Visual-Inertial Navigation, Mapping, and Loop Closure, Southern California Robotics Symposium 2016: [video] [poster]
- Relocalization and Failure Recovery for SLAM: [video]
What’s New
- February 2026: Our paper InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions was accepted to CVPR 2026.
- December 2025: We launched the Amazon Nova 2 family of foundation models: Nova 2 Lite, Pro, Omni, and Sonic, spanning reasoning, multimodal understanding and generation, and real-time conversational AI. See the press release and technical report.
- December 2024: We launched Amazon Nova, a new family of multimodal foundation models for understanding and generating text, images, and video. See the press release, AWS launch post, and technical report.
- September 2019: We released XIVO, our open-source visual-inertial odometry implementation.
Software
- XIVO (X Inertial-aided Visual Odometry) or yet another visual-inertial odometry. [code]
- VISMA dataset and utilities for our ECCV paper on Visual-Inertial Object Detection and Mapping. [code]
- GeoSup code for our ICRA paper on Geo-Supervised Visual Depth Prediction. [code]
- A minimal implementation of \(SE(3)\) (actually \(SO(3)\times \mathbb{R}^3 \) in Tensorflow for geometric learning. [code]
- A collection of PnP (Perspective-n-Point) RANSAC solvers. [code]
Theses
- Ph.D., 2019: Inertial-aided Visual Perception of Geometry and Semantics [manuscript] [slides]
- B.Eng., 2014: Robust Wide-Baseline Feature Matching for Panoramic Images
Publications
* Equal contribution.
InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions
InterPrior combines large-scale imitation pretraining with reinforcement-learning post-training to produce a versatile physics-based controller that follows sparse goals, recovers from failures, and generalizes human-object interactions to unseen settings.
Amazon Nova 2: Multimodal Reasoning and Generation Models
Nova 2 introduces four foundation models spanning configurable reasoning, native multimodal understanding and generation, and real-time speech-to-speech interaction, with context windows of up to one million tokens.
The Amazon Nova Family of Models: Technical Report and Model Card
The original Nova family pairs efficient text and multimodal understanding models with Canvas image generation and Reel video generation, emphasizing price-performance, customization, and responsible development.
Grounded Compositional and Diverse Text-to-3D with Pretrained Multi-View Diffusion Model
Grounded-Dreamer uses text-aligned four-view images as an intermediate representation, then combines sparse-view reconstruction with score distillation to generate faithful, diverse 3D assets from complex compositional prompts.
Fast Sparse-View Guided NeRF Update for Object Reconfigurations
This method detects physical scene changes from as few as four new images and uses a helper NeRF to update local geometry and appearance in one to two minutes, avoiding full scene recapture and retraining.
A Quantitative Evaluation of Score Distillation Sampling Based Text-to-3D
The work introduces human-validated metrics for the Janus problem, prompt alignment, and 3D realism, then uses the analysis to design an efficient Gaussian-splatting baseline that reduces common score-distillation artifacts.
Towards Visual Foundational Models of Physical Scenes
This paper formalizes what it means to represent a physical scene, explains why NeRFs alone cannot extrapolate beyond observed data, and explores diffusion priors as a mechanism for constructing more general visual representations.
Single View Physical Distance Estimation using Human Pose
A direct pose-based formulation jointly estimates camera intrinsics, the ground plane, and distances between people from a fixed monocular camera, enabling metric measurements without manual calibration or range sensors.
An Adaptive Framework for Learning Unsupervised Depth Completion
The framework uses reconstruction residuals to adapt visibility and regularization weights across pixels and training time, improving existing unsupervised depth-completion methods without adding trainable parameters or inference cost.
Unsupervised Depth Completion from Visual-Inertial Odometry
The method turns sparse visual-inertial landmarks into a piecewise-planar scaffold and learns dense depth through cross-modal photometric, pose, and geometric consistency; it also introduces the VOID dataset.
Geo-Supervised Visual Depth Prediction
Gravity from inertial measurements and semantic shape priors supervise monocular depth during training, encouraging horizontal and vertical surfaces to follow known geometry and improving depth prediction across datasets.
Visual-Inertial Object Detection and Mapping
An online monocular-inertial filter combines bottom-up detections with top-down object hypotheses to build a Euclidean map containing sparse geometry, recognized object models, and their poses; the work also introduces the VISMA dataset.
Poster · Video · Data · Supplement
Visual-Inertial-Semantic Scene Representation for 3D Object Detection
The system fuses visual-inertial geometry with CNN likelihoods into a persistent posterior over object identity and 3D pose, accumulating evidence over time and retaining objects through temporary occlusion.
A Simple Hierarchical Pooling Data Structure for Loop Closure
Hierarchically pooling temporally adjacent bag-of-words descriptors accelerates large-scale loop-closure search by 4–20× on average while preserving nearly the same retrieval performance as more costly schemes.
Professional Services
Reviewer for leading conferences in computer vision (CVPR, ICCV, ECCV), robotics (ICRA, IROS), and artificial intelligence (NeurIPS, ICLR, AAAI).
Talks & Workshops
- Inertial-aided Visual Perception for Localization, Mapping, and Detection, Facebook Reality Labs, Microsoft Research, and Magic Leap, 2019.
- Visual-Inertial-Semantic Scene Representation, Bridges to 3D Workshop at CVPR, 2017.
Teaching
- Teaching Assistant, CS M152A: Introductory Digital Design Laboratory, UCLA, Spring 2018.
- Teaching Assistant, Spectral Analysis of Signals, Zhejiang University. I led discussions and problem-solving sessions based in part on Linear Estimation.