Xichen Pan
Office: 424, 60 5th Ave, New York, NY 10011
Work Email: xichenpan [at] nyu [dot] edu
Personal Email: xcpan [dot] mail [at] gmail [dot] com
Bio
I am a second-year Ph.D. student of Computer Science at NYU Courant, advised by Prof. Saining Xie. I am also a Visiting Researcher at Meta AI (2024-2025 AI Mentorship Program, 20% part-time), based on NYC office. I interned at Meta GenAI Emu team (2024 Summer with Dr. Ji Hou), Microsoft Research Asia (2022-2023 with Dr. Li Dong), Alibaba Group (2022 Fall with Dr. Pengda Qin), and Horizon Robotics (2021-2022 with Yichen Gong). Previously, I obtained my bachelorβs degree in Computer Science from Shanghai Jiao Tong University (SJTU) and won the Best Thesis Award. I was fortunately advised by and maintain a close connection with Prof. Zhouhan Lin at SJTU.
Research Interest
Generative Models
Designing more controlable and high fidelity methods for image, video, and 3D generation, with a focus on:
-
Preserving spatial and temporal consistency
-
Leveraging text-to-image priors for advanced applications
Multimodal Learning
Developing vision-language models for vision-centric applications, focusing on representation learning and self-supervised pre-training
News
[07/2024] I will join Meta AI New York office as a Visiting Researcher (2024-2025 AI Mentorship Program, 20% part-time) in 2024 Fall.
[02/2024] Our paper was accepted by CVPR 2024, check it out here. See you in Seattle!
[02/2024] I will join Meta GenAI as a Research Scientist Intern in 2024 Summer. See you in Menlo Park!
[01/2024] Our paper was accepted by ICLR 2024, check it out here.
[10/2023] Our paper was accepted by WACV 2024 as Oral, check it out here.
[09/2023] Excited to start my CS Ph.D. at NYU Courant advised by Prof. Saining Xie.
[12/2022] Glad to work with Dr. Li Dong and Dr. Furu Wei at Microsoft Research Asia for the upcoming year, leading up to Fall 2023.
[06/2022] My bachelor thesis won Best Thesis Award in SJTU! Thanks my advisor Prof. Zhouhan Lin, checkout the honor roll.
[02/2022] Our paper was accepted by ACL 2022 Main Conference, check out full paper.
Education
New York University Courant Institute
Sept. 2023 -- Present
Ph.D. in Computer Science, advised by Prof. Saining Xie
Shanghai Jiao Tong University
Sept. 2018 -- June 2022
B.Eng. in Computer Science (Outstanding Graduate of Class 2022), advised by Prof. Zhouhan Lin
Publications & Manuscripts
Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs
Shengbang Tong, Ellis Brown, Penghao Wu, Sanghyun Woo, Manoj Middepogu, Sai Charitha Akula, Jihan Yang, Shusheng Yang, Adithya Jairam Iyer, Xichen Pan, Ziteng Wang, Rob Fergus, Yann LeCun, Saining Xie
NeurIPS 2024 (Oral) arXiv Code Project Page
Image Sculpting: Precise Object Editing with 3D Geometry Control
Jiraphon Yenphraphai, Xichen Pan, Sainan Liu, Daniele Panozzo, Saining Xie
CVPR 2024 arXiv Code Project Page
Kosmos-G: Generating Images in Context with Multimodal Large Language Models
Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen, Furu Wei
ICLR 2024 arXiv Code Project Page
Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models
Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, Wenhu Chen
WACV 2024 (Oral, Top 6% of accepted papers) arXiv Code
Multimodal Audio-Visual Speech Recognition System Based On Pre-trained Models
Xichen Pan
Bachelor thesis at Shanghai Jiao Tong University (Best Thesis Award, 1st/150) News Honor Roll
Leveraging Unimodal Self-Supervised Learning for Multimodal Audio-visual Speech Recognition
Xichen Pan, Peiyu Chen, Yichen Gong, Helong Zhou, Xinbing Wang, Zhouhan Lin
ACL 2022 Main Conference arXiv Code
Experience
Meta AI
Sept. 2024 β May. 2025
Visiting Researcher (2024-2025 AI Mentorship Program, 20% part-time)
Meta GenAI
May. 2024 β Sept. 2024
Research Scientist Intern
Microsoft Research Asia
Dec. 2022 β Sept. 2023
StarBridge Program Research Assistant
Alibaba Group
Sept. β Dec. 2022
Research Intern
Horizon Robotics
Apr. 2021 β July 2022
Research Intern
John Hopcroft Center for Computer Science, Shanghai Jiao Tong University
Apr. 2021 β June 2022
Research Intern
NSF Center for Big Learning, University of Florida
July β Sept. 2020
Research Intern
Selected Projects
An open-source GitHub page built for reference in selecting CS programs in north America. The page is powered by Material for MkDocs and supports collaboration through Pull Requests and GitHub Actions.
Media Exposures
Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models, Synced
Academic Service
ICML'24, ACL'24, ECCV'24, TMLR, IJCV
Some of My Friends
Cornell: Youming Deng, Gene Chou
CMU: Kexun Zhang
Georgia Tech: Haotian Xue
Neon, Inc.: Alex Chi
New York University: List of My Labmates, Hexu Zhao
Ohio State University: Kai Zhang
Oxford University: Junlin Han
Shanghai Jiao Tong University: Xinyu Xu
Stanford: Yanjie Ze
UC Berkeley: Junyi Zhang, Yichuan Wang
UMich: Yiming Dou
University of Washington: Zihan Li
USC: Di Chang