Research archive
Publications
Work on foundation models, multimodal LLMs, visual grounding, and computer vision. For the latest citation record, visit Google Scholar ↗.
32 publications
-
MANZANO: A Simple and Scalable Unified Multimodal Model with a Hybrid Vision Tokenizer
A hybrid vision tokenizer brings visual understanding and image generation into one scalable multimodal model.
-
Ferret-UI Lite: Lessons from Building Small On-Device GUI Agents
A compact 3B GUI agent for mobile, web, and desktop, trained with diverse interaction data, visual tools, and reinforcement learning.
-
Apple Intelligence Foundation Language Models: Tech Report 2025
Technical report on the foundation language models behind Apple Intelligence.
-
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms
Grounded UI understanding across phones, tablets, webpages, and TV screens through adaptive visual scaling and richer training data.
-
Improve Vision Language Model Chain-of-thought Reasoning
Rationale distillation and preference optimization strengthen reasoning in vision-language models.
-
MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA
Data, evaluation, and memory-guided modeling for understanding long egocentric videos.
-
Contrastive Localized Language-Image Pre-Training
Region-text contrastive learning produces visual representations for fine-grained recognition, retrieval, and grounded multimodal understanding.
-
Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models
A study of synthetic captions and original web text for pre-training contrastive, language, and diffusion models.
-
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning
Data and training recipes for multimodal models with stronger text-rich image understanding, grounding, and multi-image reasoning.
-
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
Referring, grounding, and reasoning capabilities for understanding mobile user interfaces.
-
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training
A systematic study of architecture and data choices for scaling multimodal language-model pre-training.
-
How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts
An empirical evaluation of multimodal models under misleading visual questions and prompts.
-
Empowering Unsupervised Domain Adaptation With Large-Scale Pre-Trained Vision-Language Models
Efficient adaptation and domain-aware pseudo-labeling transfer large vision-language models across visual domains.
-
GLIPv2: Unifying Localization and Vision-Language Understanding
One model connects object localization with vision-language tasks, from open-vocabulary detection to visual question answering.
-
DIOR: DIstill Observations to Representations for Multi-Object Tracking and Segmentation
Spatial and temporal feature distillation supports joint multi-object tracking and segmentation.
-
Monocular 3D Localization of Vehicles in Road Scenes
A monocular framework combines 3D detection, tracking, and temporal optimization to localize vehicles.
-
U3D-MOLTS: Unified 3D Monocular Object Localization, Tracking and Segmentation
A unified pipeline for monocular 3D object localization, multi-object tracking, and segmentation.
-
HVPS: A Human Video Panoptic Segmentation Framework
A human tracking and panoptic segmentation pipeline built around appearance features and temporal association.
-
ROD2021 Challenge: A Summary for Radar Object Detection Challenge for Autonomous Driving Applications
Benchmark and challenge results for detecting road objects directly from radar observations.
-
IA-MOT: Instance-Aware Multi-Object Tracking with Motion Consistency
Instance-level appearance and motion consistency help track objects across static and moving cameras.
-
LIFTS: LiDAR and Monocular Image Fusion for Multi-Object Tracking and Segmentation
Fusing LiDAR and monocular imagery supports object detection, tracking, and segmentation.
-
Bundle Adjustment for Monocular Visual Odometry Based on Detections of Traffic Signs
Detected traffic signs constrain camera poses and map points to reduce monocular odometry drift.
-
Eye in the Sky: Drone-Based Object Tracking and 3D Localization
Object tracking and 3D localization from video captured by a single camera on a drone.
-
VisDrone-MOT2019: The Vision Meets Drone Multiple Object Tracking Challenge Results
Results and analysis from the VisDrone 2019 challenge on multi-object tracking in drone video.
-
Bundle Adjustment for Monocular Visual Odometry Based on Detected Traffic Sign Features
Traffic-sign features provide additional geometric constraints for monocular visual odometry.
-
Exploit the Connectivity: Multi-Object Tracking with TrackletNet
Learned connections between short tracklets combine appearance and temporal information for multi-object tracking.
-
Electro-thermal Simulation of Graphene Nanoribbon Interconnects
Electrothermal modeling of graphene nanoribbon interconnects.
No publications match these filters.