Jun-Yan Zhu

• Interaction between creators and generative models: How can we help creators control the model outputs more easily? We develop algorithms and interfaces for controllable visual synthesis (e.g., images, videos, 3D, visual+tactile)
• Rewriting and searching generative models: How can creators repurpose existing models for new tasks, concepts, and styles? How could they rewrite the rules of models? Which model shall they use as a starting point?
• Co-existence of creators and generative models: How can we allow creators to opt in or out of generative models at any time? If opting in, how do we credit creators for contributing training data?
• Synthetic data generation with generative models: How can we use generative models to produce useful data for improving computer vision and robotics systems?

Biography: 

Jun-Yan Zhu is an Assistant Professor at CMU’s School of Computer Science. Prior to joining CMU, he was a Research Scientist at Adobe Research and a postdoctoral researcher at MIT CSAIL. He obtained his Ph.D. from UC Berkeley and his B.E. from Tsinghua University. He studies computer vision, computer graphics, and computational photography. 

He has received the Samsung AI Research of the Year (2024), the Packard Fellowship (2023), the NSF CAREER Award (2023), the ACM SIGGRAPH Outstanding Doctoral Dissertation Award (2018), the UC Berkeley EECS David J. Sakrison Memorial Prize for outstanding doctoral research (2018), and several faculty awards (JPMC, Amazon, Sony, Cisco).  His co-authored work has received the 2024 ICRA Best Paper on Human-Robot Interaction, SIGGRAPH 2019 Real-time Live Best of Show Award and Audience Choice Award, the 100 Greatest Innovations of 2019 by Popular Science, and NVIDIA Pioneer Research Award (2018).  His work and commentary have been covered in the New Yorker, the New York Times, BBC, CNN, Reuters, and The Economist.