Open to Research Opportunities

Xinyao Zhang 张欣瑶

Tsinghua University · THU-CVML Beijing, China xinyao-z24@mails.tsinghua.edu.cn

I am a 2nd-year Master's student at Tsinghua University (Bio3 program, Precision Medicine and Public Health), affiliated with the Computer Vision and Machine Learning Lab (THU-CVML). My research focuses on generative visual models, instruction-guided video editing, and visual place recognition. I am currently an intern at ByteDance Douyin. Previously, I was a research intern at Baidu's Digital Human and Video Generation Group until 2026.6, working on video generation and controllable generation.

Xinyao Zhang
Xinyao Zhang
M.S. Student @ Tsinghua · Intern @ ByteDance Douyin
3
Papers
2
First-Author
210⭐
GitHub Stars
CCF-A
Best Venue
01

Research Interests

🎬
Instruction-Guided Video Editing
Building diffusion-based models that faithfully follow text instructions while preserving temporal consistency and motion dynamics across video frames.
✨
Generative Visual Models
Exploring controllable generation, semantic disentanglement, and unified generation frameworks for images and videos using latent diffusion models.
🗺️
Visual Place Recognition
Designing compact, discriminative global image descriptors for robot localization, with a focus on cross-domain generalization and efficient feature aggregation.
02

Publications

🎉 ECCV 2026
SAMA: Factorized Semantic Anchoring and Motion Alignment for Instruction-Guided Video Editing
Xinyao Zhang1,2,*, Wenkai Dong2,*, Yuxin Song2,*,†, Bo Fang2,3, Qi Zhang2, Jing Wang1,2, Fan Chen2, Hui Zhang2, Haocheng Feng2, Yu Lu4,†, Hang Zhou2, Chun Yuan1, Jingdong Wang2
*Equal contribution   †Corresponding author
1Tsinghua University, 2Baidu Inc, 3City University of Hong Kong, 4Zhejiang University
We address the fundamental challenge in instruction-guided video editing: simultaneously achieving precise semantic modification and faithful motion preservation. SAMA factorizes video editing into two sub-tasks — Semantic Anchoring (jointly predicting semantic tokens and video latents on sparse anchor frames to establish reliable visual anchpoints) and Motion Alignment (pre-training on motion-centric video inpainting tasks including cube inpainting, speed perturbation, and tube shuffle). A two-stage training strategy enables learning from unpaired data in Stage 1, achieving strong zero-shot editing, before supervised fine-tuning in Stage 2. SAMA achieves state-of-the-art among open-source methods on VIE-Bench, OpenVE-Bench, and ReCo-Bench, competitive with commercial systems like Kling-Omni.
Video Editing Diffusion Models Semantic Anchoring Motion Alignment SAMA-14B 210⭐ GitHub
2026
Co-First Author
⭐ NeurIPS 2024 · CCF-A
SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition
(Co-author including Xinyao Zhang)
THU-CVML, Tsinghua University
Visual place recognition (VPR) under domain shifts remains challenging due to high-dimensional descriptors and distribution bias in traditional NetVLAD. SuperVLAD proposes a cluster-center-free framework that eliminates inter-domain distribution shift, achieving superior cross-domain generalization. With as few as 4 clusters, SuperVLAD matches or exceeds standard NetVLAD with many more clusters, significantly reducing descriptor dimensionality while achieving state-of-the-art performance on Pitts30k, MSLS-val, and multiple VPR benchmarks.
Visual Place Recognition Global Descriptors VLAD Cross-Domain Image Retrieval
2024
Co-Author
🔵 ICASSP 2026 · CCF-B
3GeM Pooling: Direction-Aware and Compact Global Descriptors for Visual Place Recognition
Xinyao Zhang et al.
THU-CVML, Tsinghua University
Standard GeM pooling treats all spatial locations uniformly, inadvertently incorporating non-discriminative regions (sky, pedestrians, dynamic objects) into global descriptors. We propose 3GeM Pooling, a direction-aware aggregation method that performs generalized mean pooling across three dimensions (H×W, H×D, W×D), capturing richer structural information including horizontal and vertical directional differences. The SGCE (Spatially Guided Channel Enhancement) module uses spatial cues to generate attention weights for channel features, boosting discriminability while maintaining compact 1024-dimensional representations. Achieves state-of-the-art on MSLS, Pitts30k, and other benchmarks with better generalization in low-data regimes.
Visual Place Recognition GeM Pooling Direction-Aware Compact Descriptors SGCE SOTA
2026
First Author
03

Education

Tsinghua University
M.S. in Precision Medicine & Public Health (Bio3) · THU-CVML Lab · Supervisor: Prof. Chun Yuan
2024 – Present
Qingdao University
B.S. in Applied Statistics (Mathematics) · GPA Top of Department · First student admitted to Tsinghua via recommendation
2020 – 2024
04

Skills & Tools

Deep Learning Frameworks
PyTorch PaddlePaddle Diffusers Transformers DiffSynth
Research Focus
Diffusion Models Video Generation Video Editing Image Retrieval Feature Aggregation
Engineering
Multi-GPU Training Model Fine-tuning Ablation Study Result Visualization HuggingFace Hub
Programming & Math
Python C++ (OI) Statistics Linear Algebra Probability Theory
05

Experience & Awards

Intern · ByteDance Douyin
Douyin · Generative Visual Algorithm Research
2026.6 – Present
Research Intern · Baidu Inc
Digital Human & Video Generation Group · Generative Visual Algorithm Research · Video editing, controllable generation, unified generation frameworks, post-training on Wan series models
2025 – 2026.6
Honors & Awards
Provincial 2nd Prize (NOIP) · National-level Algorithm & Modeling Competition Awards · Outstanding Graduate (Qingdao University) · Top Academic Award (President's Scholarship) · Learning Model Award
2016 – 2024