MSR Thesis Talk: Chonghyuk Song
Title: Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesis Abstract: We explore the task of embodied view synthesis from monocular videos of deformable scenes. Given a minute-long RGBD video of people interacting with their pets, we render the scene from novel camera trajectories derived from in-scene motion of actors: (1) egocentric cameras that simulate the point [...]
Design Iteration of Dexterous Compliant Robotic Manipulators
Abstract: One goal of personal robotics is to have robots in homes performing everyday tasks efficiently to improve our quality of life. Towards this end, manipulators are needed which are low cost, safe around humans, and approach human-level dexterity. However, existing off-the-shelf manipulators are expensive both in cost and manufacturing time, difficult to repair, and [...]
MSR Thesis Talk: Shivam Duggal
Title: Learning Single Image 3D Reconstruction from Single-View Image Collections Abstract We present a framework for learning 3D object shapes and dense cross-object 3D correspondences from just an unaligned category-specific image collection. The 3D shapes are generated implicitly as deformations to a category-specific signed distance field and are learned in an unsupervised manner solely from unaligned [...]
Whisker Sensors for Unstructured Environments
Abstract: As robot applications expand from controllable factory settings to unknown environments, the robots will need a larger breadth of sensors to perceive these complex environments. In this thesis, I focus on developing whisker sensors for robot perception. The inspiration for whisker sensors comes from the biological world, where whiskers serve as tactile and flow [...]
MSR Thesis Talk: Himangi Mittal
Title: Audio-Visual State-Aware Representation Learning from Interaction-Rich Data Abstract In robotics and augmented reality, the input to the agent is a long stream of video from the first-person or egocentric point of view. Recently, there have been significant efforts to capture humans from their first-person/egocentric view interacting with their own environment as they go about [...]
MSR Thesis Talk: Ken Liu
Title: On Privacy and Personalization in Federated Learning: Analyses and Applications Abstract: Recent advances in machine learning often rely on large and centralized datasets. However, curating such data can be challenging when they hold private information, and policies/regulations may mandate that they remain distributed across data silos (e.g. mobile devices or hospitals). Federated learning (FL) [...]
Carnegie Mellon University
MSR Thesis Talk: Haolun Zhang
Title: Seeing in 3D: Towards Generalizable 3D Visual Representations for Robotic Manipulation Abstract: Despite the recent progress in computer vision and deep learning, robot perception remains a tremendous challenge due to the variations of the objects and the scenes in manipulation tasks. Ideally, a robot trying to manipulate a new object should be able to [...]
MSR Thesis Talk: Muyang Li
Title: Efficient Spatially Sparse Inference for Conditional GANs and Diffusion Models Abstract: During image editing, existing deep generative models tend to re-synthesize the entire output from scratch, including the unedited regions. This leads to a significant waste of computation, especially for minor editing operations. In this work, we present Spatially Sparse Inference (SSI), a general-purpose technique [...]
3D-aware Conditional Image Synthesis
Abstract: We propose pix2pix3D, a 3D-aware conditional generative model for controllable photorealistic image synthesis. Given a 2D label map, such as a segmentation or edge map, our model learns to synthesize a corresponding image from different viewpoints. To enable explicit 3D user control, we extend conditional generative models with neural radiance fields. Given widely-available posed [...]
Promoting Human Creativity with FRIDA: Framework and Robotics Initiative for Developing Arts
ABSTRACT: FRIDA, a reference to the vibrant painter Frida Kahlo, stands for a Framework and Robotics Initiative for Developing Arts to promote human creativity. FRIDA supports intuitive ways for people to collaboratively create artworks including natural language, images, and sounds. Because FRIDA is for real-world arts, our work is uniquely different from digital art tools [...]
RI Faculty Business Meeting
Meeting for RI Faculty. Discussions include various department topics, policies, and procedures. Generally meets weekly.
Robotic Climbing for Extreme Terrain Exploration
Abstract: Climbing robots can investigate scientifically valuable sites that are inaccessible to conventional rovers due to steep terrain features. Robots equipped with microspine grippers are particularly well-suited to ascending rocky cliff faces, but existing designs are either large and slow, or limited to relatively flat surfaces such as buildings. We have developed a novel free-climbing [...]
MSR Thesis Talk: Rohan Zeng
Title: Spectral Unmixing and Mapping of Coral Reef Benthic Cover Abstract: Coral reefs are important to the global ecosystem and the local communities and wildlife that rely on the habitat they create. However, coral reefs are also in critical and rapid decline: reefs have degraded over recent decades and what remains is at increasing risk [...]
Generative modeling: from 3D scenes to fields and manifold
Abstract: In this keynote talk, we delve into some of our progress on generative models that are able to capture the distribution of intricate and realistic 3D scenes and fields. We explore a formulation of generative modeling that optimizes latent representations for disentangling radiance fields and camera poses, enabling both unconditional and conditional generation of 3D [...]