Lei Ke

Senior Researcher at Tencent AI Seattle

I am a Senior Research Scientist in Tencent AI, Seattle Lab. My primary research interest lies in building multimodal foundation systems, especially the visual understanding, reasoning, and generation. Previously, I worked as a Postdoctoral Research Associate at Computer Science of Carnegie Mellon University and in the Computer Vision Lab of ETH Zurich. I obtained my Ph.D. degree from CSE Department at HKUST in mid 2023, supervised by Chi-Keung Tang and Yu-Wing Tai. During the PhD journey, I also spent two years as a visiting scholar at ETH Zurich.I received my B.E. degree from the school of computer science at Wuhan University.

More info: [Email],[Google Scholar],[X],[GitHub], where my opensource projects obtains over 10K+ GitHub stars.


Updates


Selected Publications

Segmentation and Tracking:
Segment Anything in High Quality
NeurIPS 2023
Lei Ke*, Mingqiao Ye*, Martin Danelljan, Yifan Liu, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu
(* denotes equal contribution)
We propose HQ-SAM to upgrade SAM for high-quality zero-shot segmentation.
HQ-SAM receives 2000+ Github stars in one month.
Gaussian Grouping: Segment and Edit Anything in 3D Scenes
ECCV 2024
Mingqiao Ye, Martin Danelljan, Fisher Yu, Lei Ke
( denotes Project Lead)
Gaussian Grouping for open-world 3D Anything reconstruction, segmentation and editing.
[Paper] [Project] [Web]
Matching Anything By Segmenting Anything
CVPR 2024
Siyuan Li, Lei Ke, Martin Danelljan, Luigi Piccinelli, Mattia Segu, Luc Van Gool, Fisher Yu
MASA provides a universal instance appearance model for matching any objects in any domain.
Highlight (~3% acceptance rate)
Segment Anything Meets Point Tracking
arXiv
Frano Rajič, Lei Ke, Yu-Wing Tai, Chi-Keung Tang, Martin Danelljan, Fisher Yu
We propose SAM-PT to extend SAM to zero-shot video segmentation with point-based tracking.
SAM-PT receives 500+ Github stars in one week.
Mask Transfiner for High-Quality Instance Segmentation
CVPR 2022
Lei Ke, Martin Danelljan, Xia Li, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu
An efficient transformer-based method for highly accurate instance segmentation.
Transfiner receives 300+ Github stars in 3 months.
Prototypical Cross-Attention Networks for Multiple Object Tracking and Segmentation
NeurIPS 2021
Lei Ke, Xia Li, Martin Danelljan, Yu-Wing Tai, Chi-Keung Tang, Fisher Yu
Efficient cross-attention on space-time memory for video instance segmentation.
Spotlight (3% acceptance rate). PCAN receives 200+ Github stars in one month.
Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers
CVPR 2021 & TPAMI 2023
Lei Ke, Yu-Wing Tai, Chi-Keung Tang
Instance segmentation with bilayer decoupling structure for occluder & occludee.
BCNet receives 300+ Github stars in 6 months.
3D Reconstruction, Tracking and Generation:
TAPIP3D: Tracking Any Point in Persistent 3D Geometry
arXiv 2025
Bowei Zhang*, Lei Ke*, Adam W. Harley, Katerina Fragkiadaki
(* denotes equal contribution)
TAPIP3D: Long-term feed-forward 3D point tracking in persistent 3D point maps.
Video Depth without Video Models
CVPR 2025
Bingxin Ke, Dominik Narnhofer, Shengyu Huang, Lei Ke, Torben Peters, Katerina Fragkiadaki, Anton Obukhov, Konrad Schindler
RollingDepth: A universal monocular depth estimator for arbitrarily long videos.
DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos
NeurIPS 2024
Wen-Hsuan Chu*, Lei Ke*, Katerina Fragkiadaki
DreamScene4D for generating 3D dynamic scenes of multiple objects from monocular videos.
[Paper] [Project] [Web]

Experiences

2024.04—2025.03: Postdoc at MLD, CMU

2023.07—2024.03: Postdoc at CVL, ETHz

2021.01—2023.03: Visiting PhD student at CVL, ETHz

2019.05—2023.05: HKUST Computer Vision Research Assistant

2017.11—2019.11: Tencent Youtu X-lab Computer Vision Research Intern, worked closely with Wenjie Pei.

2017.05—2017.10: Alibaba Engineering Intern

2016.05—2017.02: Undergraduate Research Assistant at Wuhan University


Awards

2023, Received CVPR Doctoral Consortium Awards(13% success rate), supervised by Michael Maire.
2022, Most Popular Speakers in TechBeat
2022, Research Travel Grant, ETH Zürich
2019, Research Travel Grant, HKUST
2019 - Present, Postgraduate Studentship, HKUST
2017, COMAP's Mathematical Contest in Modeling, Honorable prize
2015 - 2017, Excellent Student Scholarship, Wuhan University
2016, National Software Design Competition, Second Prize
2016, National Inspirational Scholarship, Wuhan University
2015, National College Students' Mathematics Competition, Third Prize

Professional Activities


Misc